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Economic  Development  Division 


B-283391 
June  28,  2000 

The  Honorable  John  J.  Duncan,  Jr. 

Chairman 

The  Honorable  William  0.  Lipinski 
Ranking  Democratic  Member 
Subcommittee  on  Aviation 
Committee  on  Transportation 
and  Infrastructure 
House  of  Representatives 

As  requested,  we  are  reporting  on  the  actions  taken  by  the  Federal  Aviation  Administration  (FAA), 
through  the  Safer  Skies  initiative,  towards  the  goal  of  reducing  the  nation’s  fatal  aviation  accident 
rates  by  2007.  Our  report  contains  recommendations  designed  to  ensure  that  the  implementation  of 
interventions  approved  by  FAA  and  the  Safer  Skies  steering  committees  is  tracked  and  that  the 
interventions  are  evaluated  for  their  effectiveness  in  meeting  the  goal. 

As  arranged  with  your  offices,  unless  you  publicly  announce  its  contents  earlier,  we  plan  no  further 
distribution  of  this  report  until  7  days  after  the  date  of  this  letter.  At  that  time,  we  will  send  copies  to  o 
the  appropriate  congressional  committees;  the  Honorable  Rodney  E.  Slater,  Secretary  of 
Transportation;  the  Honorable  Jane  F.  Garvey,  Administrator,  FAA;  and  the  Honorable  Jacob  Lew, 
Director,  Office  of  Management  and  Budget.  We  will  also  make  copies  available  to  others  upon 
request. 
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B-283391 


If  you  have  any  questions  about  this  report  please  contact  me  or  Robert  White  at  (202)  512-2834. 
Other  key  contributors  to  this  report  are  listed  in  appendix  I. 

Sincerely  yours, 


Gerald  L.  Dillingham,  Ph.D. 
Associate  Director,  Transportation 
Issues 


Page  6 


GAO/RCED-OO -111  Safer  Skies  Initiative 


Page  7 


GAO/RCED-OO-111  Safer  Skies  Initiative 


Executive  Summary 


Purpose 


The  continued  growth  forecast  for  U.S.  aviation  in  the  coming  decade  will 
likely  bring  a  rise  in  fatal  accidents  if  the  current  accident  rate  is  not 
reduced.1  Commercial  aviation,  used  by  most  Americans  when  they  fly, 
experienced  an  average  of  6  fatal  accidents  a  year  in  the  United  States  in 
1994-96;  general  aviation  experienced  an  average  of  380  a  year.2  If  the 
projected  growth  in  flight  hours  occurs  and  the  fatal  accident  rate  is  not 
reduced,  GAO  estimates  in  this  report  that  the  number  of  fatal  commercial 
aviation  accidents  could  rise  to  9  per  year  and  the  number  of  fatal  general 
aviation  accidents  to  484  by  2007.  The  Federal  Aviation  Administration 
(FA A),  the  Congress,  and  the  aviation  industry  have  acknowledged  this 
potential  danger  and  have  recommended  ways  to  address  it.  In  1997,  two 
major  commissions  on  aviation  safety  recommended  reducing  the  nation’s 
aviation  accident  rate  by  80  percent  by  2007.  To  meet  this  challenging  goal, 
both  the  White  House  Commission  on  Aviation  Safety  and  Security  and  the 
congressionally  mandated  National  Civil  Aviation  Review  Commission 
recommended  that  FAA  and  the  aviation  industry  work  together  to  identify 
and  address  the  causes  of  fatal  accidents.  To  unify  government  and 
industry  efforts  to  reduce  the  accident  rate  by  addressing  the  greatest 
threats  to  aviation  safety,  FAA  announced  the  Safer  Skies  initiative  in  April 
1998  with  the  broad  initial  goal  of  reducing  the  number  of  fatal  accidents 
per  million  flight  hours  by  80  percent  by  2007. 

Aviation  experts  from  FAA,  the  aviation  industry,  and  other  government 
agencies  formed  three  steering  committees  to  oversee  the  initiative’s  work 
in  three  broad  areas:  commercial  aviation,  general  aviation,  and  cabin 
safety  The  steering  committees  will  analyze  data  to  identify  the  most 
serious  threats  to  safety,  to  find  the  root  causes  of  accidents,  and  then  to 
determine  the  best  actions  to  break  the  chain  of  events  that  lead  to 
accidents.  Resources  will  be  directed  first  to  implementing  those  best 
actions,  referred  to  as  interventions.  The  steering  committees  have 
identified  16  specific  safety  problems — 6  related  to  commercial  aviation,  6 
to  general  aviation,  and  4  to  cabin  safety.  These  problems  will  be  addressed 


‘The  fatal  accident  rate  is  calculated  by  dividing  the  number  of  fatal  accidents  by  a  measure 
of  aviation  activity  such  as  the  number  of  aircraft  hours  flown. 

Commercial  aviation  includes  both  large  air  carrier  operations  and  smaller  commuter 
operations.  General  aviation  includes  a  wide  variety  of  aircraft,  ranging  from  corporate  jets 
to  small  piston-engine  aircraft  as  well  as  helicopters,  gliders,  and  aircraft  used  in  operations 
such  as  firefighting  and  agricultural  spraying.  In  establishing  accident  reduction  goals,  FAA 
and  Safer  Skies  steering  committees  used  1994-96  as  the  baseline  years  for  commercial 
aviation  and  1996-98  as  the  baseline  years  for  general  aviation. 
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by  teams  of  aviation  experts  who  can  recommend  one  or  many 
interventions  for  the  safety  problems  they  are  addressing.  Some  of  the 
safety  problems,  such  as  weather,  will  be  addressed  by  both  commercial 
aviation  and  general  aviation  teams  because  their  causes  and  interventions 
may  differ  for  these  types  of  operations. 

In  light  of  the  critical  importance  of  the  Safer  Skies  initiative  in  improving 
aviation  safety,  the  Chairman  and  Ranking  Democratic  Member  of  the 
Subcommittee  on  Aviation,  House  Committee  on  Transportation  and 
Infrastructure,  asked  GAO  to  review  the  implementation  of  this  initiative. 
Specifically,  they  asked  GAO  to  determine  (1)  to  what  extent  addressing 
the  safety  problems  selected  by  the  Safer  Skies  initiative  will  help  reduce 
the  fatal  accident  rate;  (2)  what  progress  the  initiative  has  made  in 
identifying  and  implementing  interventions  to  address  each  of  these  safety 
problems;  (3)  what  progress  has  been  made  in  assessing  the  effectiveness 
of  those  interventions;  and  (4)  how  FAA  is  coordinating  the  Safer  Skies 
initiative  with  other  safety  activities  conducted  throughout  the  agency,  in 
partnership  with  the  aviation  industry,  and  by  other  federal  agencies. 


Background 


Since  1982,  air  travel  in  the  United  States  has  increased  dramatically,  and 
flight  safety  has  improved.  The  number  of  hours  flown  by  commercial 
aircraft  more  than  doubled  from  8  million  hours  in  1982  to  nearly  18  million 
hours  in  1999.  FAA  estimates  that  commercial  aviation  aircraft  will  fly  more 
than  24  million  hours  in  2007,  an  increase  of  37  percent  from  1999.  Growth 
in  general  aviation  has  been  less  consistent,  but  FAA  estimates  that  general 
aviation  flight  hours  will  increase  to  about  36  million  hours  in  2007,  a 
growth  of  nearly  19  percent  over  1999.  Although  the  accident  rates  for  both 
types  of  operations  are  low,  both  the  number  and  the  frequency  of  aviation 
deaths  will  likely  increase  if  these  rates  are  not  reduced  as  the  growth  in  air 
travel  continues.  In  the  10-year  period  from  1988  through  1997,  the  United 
States  had  4,471  fatal  aviation  accidents  that  resulted  in  a  total  of  9,802 
deaths.  Commercial  aviation  accounted  for  only  2  percent  of  the  fatal 
accidents,  while  general  aviation  accounted  for  98  percent.  (See  table  1.) 
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Results  in  Brief 


Table  1 :  Number  of  Fatal  Accidents  and  Deaths  by  Type  of  Aviation  Operation, 
1988-97 


Fatal  accidents  Deaths 


Type  of  operation 

Number 

Percentage 

Number 

Percentage 

Commercial  aviation 

85 

2 

1,756 

18 

General  aviation 

4,386 

98 

8,046 

82 

Total 

4,471 

100 

9,802 

100 

Source:  GAO’s  analysis  of  data  from  the  National  Transportation  Safety  Board. 


The  Safer  Skies  initiative  addresses  the  safety  problems  that  have 
contributed  to  fatal  accidents  in  the  past,  and  in  conjunction  with  other 
safety  programs,  it  can  be  expected  to  reduce  the  fatal  accident  rate  and 
thus  enhance  the  safety  of  the  nation’s  air  passengers.  In  commercial 
aviation,  the  initiative  addresses  safety  problems  that  accounted  for  over 
three-quarters  of  the  fatal  accidents  in  those  operations  in  1988-97.  Other 
FAA  initiatives  are  addressing  additional  safety  problems,  which  should 
complement  Safer  Skies’  efforts  to  meet  the  goal  of  an  80-percent  reduction 
in  the  fatal  accident  rate  for  this  segment  of  the  aviation  industry.  In 
general  aviation,  the  Safer  Skies  initiative  plans  to  address  safety  problems 
that  appear  to  be  the  most  common  causes  of  fatal  accidents.  The  initiative 
has  adopted  a  less  aggressive  goal  in  general  aviation  of  reducing  the 
number  of  fatal  accidents  to  350  in  2007,  which  represents  about  a  20- 
percent  reduction.  Finally,  the  initiative  addressed  four  safety  problems  in 
cabin  safety.  Improving  cabin  safety  will  have  little  impact  on  lowering  the 
fatal  accident  rate  because  cabin  safety  accounted  for  only  two  U.S. 
commercial  aviation  fatalities  in  1988-97.  No  quantitative  goal  was  set  for 
safety  improvements  in  cabin  safety.  To  date,  safety  improvement  efforts  by 
FAA  and  the  initiative  have  focused  on  reducing  the  causes  of  past 
accidents  and  incidents,  which  may  not  be  entirely  predictive  of  future 
ones.  Studying  growth  and  technological  changes  in  the  aviation  industry 
can  help  anticipate  and  prevent  the  safety  problems  and  accidents  that  are 
likely  to  arise  from  such  changes.  An  international  work  group  has  been 
formed  to  address  future  hazards,  and  a  number  of  FAA  staff  participate  in 
this  work  group.  Coordinating  these  efforts  with  Safer  Skies’  work  should 
enhance  the  initiative’s  efforts  to  reduce  the  fatal  accident  rate. 

As  of  April  1,  2000,  Safer  Skies  teams  had  started  work  on  13  of  the  16 
safety  problems  and  had  begun  implementing  interventions  for  5  of  these — 
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2  in  commercial  aviation  and  3  in  cabin  safety.  Teams  have  made  the  most 
progress  in  selecting  interventions  to  address  safety  problems  when  they 
have  been  able  to  build  on  previous  studies  for  which  widely  supported 
recommendations  already  existed.  Since  its  inception  in  April  1998,  the 
Safer  Skies  initiative  has  evolved  as  new  safety  problems  have  been 
addressed.  For  example,  the  process  used  to  analyze  safety  problems  and 
select  interventions  has  been  modified  as  Safer  Skies  teams  have  begun  to 
address  safety  problems  that  have  received  less  extensive  study.  Because 
many  of  these  safety  problems  are  long-standing  ones  that  have  not  been 
fully  resolved  by  prior  efforts,  progress  will  depend  on  effectively 
implementing  the  chosen  interventions.  The  initiative  has  developed  a 
process  for  tracking  the  implementation  of  interventions  to  improve  safety 
in  commercial  aviation.  However,  the  implementation  of  Safer  Skies’ 
interventions  is  not  assured  because  the  tracking  system  for  commercial 
aviation  is  not  sufficiently  detailed  to  assess  progress  in  implementing 
interventions,  the  system  for  general  aviation  is  still  in  development,  and  no 
system  exists  for  tracking  recommended  interventions  in  cabin  safety. 
Without  such  systems,  the  Safer  Skies  initiative  cannot  ensure  that  all  of 
the  interventions  approved  to  increase  aviation  safety  will  be  put  into 
action. 

Since  most  of  the  interventions  developed  under  the  Safer  Skies  initiative 
are  in  early  implementation  stages,  little  progress  has  been  made  in 
evaluating  their  effectiveness.  Of  the  five  Safer  Skies  teams  that  have  begun 
implementing  interventions,  only  one  has  developed  a  performance 
measure  to  evaluate  whether  the  interventions  it  has  selected  are  helping  to 
reduce  the  safety  problems  that  cause  fatal  accidents  and  are  worth  what 
they  cost.  Such  evaluation  requires  that  performance  measures  be 
developed  to  serve  as  the  yardsticks  for  measuring  the  progress  made 
toward  the  program  goals — a  process  required  for  federal  programs  by  the 
Government  Performance  and  Results  Act  of  1993.  The  initiative  has 
established  safety  goals  based  on  reducing  the  fatal  accident  rate  for 
commercial  aviation  by  80  percent  and  reducing  the  number  of  fatal 
general  aviation  accidents  to  350  by  2007.  It  plans  to  measure  annual 
progress  toward  meeting  those  goals.  Fatal  accidents  occur  too  rarely, 
especially  in  commercial  aviation,  to  serve  as  measures  of  the  effectiveness 
of  specific  interventions.  Other  indicators  exist  or  can  be  developed  to 
measure  the  unique  effect  of  individual  interventions. 

FAA  has  coordinated  extensively  with  aviation  experts  from  industry,  other 
federal  government  agencies,  and  its  own  staff,  but  GAO’s  review  identified 
three  coordination  problems  that  could  undermine  the  implementation  and 
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evaluation  of  Safer  Skies’  interventions.  First,  although  FAA  officials  have 
repeatedly  committed  to  funding  interventions  agreed  upon  by  all  parties 
working  on  the  initiative,  skepticism  still  exists  among  some  participants 
as  to  whether  this  commitment  can  or  will  be  honored.  This  is  particularly 
true  in  general  aviation.  Furthermore,  if  funding  is  limited,  it  remains 
unclear  what  process  will  be  used  to  reprioritize  available  resources  to 
ensure  funding  for  interventions  that  emerge  later  but  have  greater 
potential  for  reducing  the  fatal  accident  rate.  Finally,  the  Safer  Skies 
initiative,  FAA,  and  the  Department  of  Transportation  (DOT)  have  not 
agreed  on  how  they  will  measure  progress  in  achieving  the  accident 
reduction  goal  for  commercial  aviation. 

We  presented  a  draft  of  this  report  to  DOT  and  FAA  for  comment.  DOT  and 
FAA  officials  characterized  the  report  as  fair  and  reasonable  and  provided 
technical  clarifications,  which  were  made  as  appropriate.  The  officials 
concurred  with  most  of  our  recommendations.  However,  DOT  and  FAA 
officials  disagreed  with  our  recommendation  that  some  basic  criteria  and 
processes  should  be  established  for  reprioritizing  interventions  if  funding  is 
limited.  Because  we  believe  that  such  criteria  and  processes  would  be 
useful  in  assessing  the  potential  impact  of  safety  interventions,  we  did  not 
modify  or  delete  our  recommendation.  The  officials  also  disagreed  with 
two  of  our  recommendations  calling  for  an  analyses  of  safety  problems  that 
have  not  been  studied  previously  and  of  problems  that  may  arise  in  the 
future.  The  officials  provided  information  showing  that  FAA  is  taking  these 
actions,  so  we  withdrew  these  recommendations. 
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Principal  Findings 


The  Safer  Skies  Initiative 
Should  Help  Improve 
Aviation  Safety 


The  six  safety  problems  that  Safer  Skies  addresses  in  commercial  aviation 
accounted  for  about  79  percent  of  the  fatal  accidents  in  commercial 
aviation  in  1988-97.  Three  of  these  safety  problems  accounted  for  58  of  the 
85  accidents  during  this  period:  pilots’  losing  control  of  their  aircraft,  pilots’ 
flying  otherwise  controllable  aircraft  into  the  ground  or  water,  and 
accidents  during  approach  and  landing.3  Commercial  aviation  teams  will 
develop  interventions  for  these  safety  problems  in  large  aircraft,  while 
accidents  that  involved  smaller  commuter  aircraft  were  referred  to  the 
general  aviation  steering  committee  for  review.  To  further  reduce  the  fatal 
accident  rate  for  commercial  aviation,  the  commercial  aviation  teams  will 
address  three  other  safety  problems4  that  resulted  in  fewer  fatal  accidents 
but  have  the  potential  to  cause  many  more  in  the  future.  FAA  also  has 
ongoing  initiatives  to  address  the  causes  of  several  additional  safety 
problems  that  caused  four  fatal  accidents,  including  sabotage,  fuel  tank 
explosions,  and  structural  problems. 

In  general  aviation,  the  Safer  Skies  initiative  also  addresses  major  safety 
problems,  but  the  goal  chosen  does  not  encourage  aggressive  steps  to 
decrease  general  aviation  accidents.  Although  the  data  available  on  general 
aviation  accidents  are  less  detailed  than  those  available  on  commercial 
aviation  accidents,  the  problems  to  be  addressed  in  general  aviation 
include  ones  identified  in  past  safety  reports  and  in  the  National 
Transportation  Safety  Board’s  (NTSB)  accident  reports  as  major  causes  of 


3Loss  of  control,  which  refers  to  accidents  in  which  the  pilot  should  have  maintained  or 
regained  control  of  the  aircraft  but  did  not,  is  the  largest  cause  of  fatal  large  air  carrier 
accidents  in  the  United  States.  Controlled  flight  into  terrain,  which  refers  to  flying  an 
otherwise  controllable  aircraft  into  the  ground  or  water,  is  the  leading  cause  of  fatal  large  air 
carrier  accidents  worldwide.  Approach  and  landing  refers  to  situations  in  which  a  crash 
occurs  during  the  approach  to  the  airport  or  attempt  to  land  when  the  pilot  might  have  been 
able  to  land  safely  but  did  not. 

These  problems,  which  resulted  in  a  total  of  nine  fatal  accidents,  include  weather, 
uncontained  engine  failure,  and  runway  incursions.  Uncontained  engine  failure  occurs  when 
a  heavy  engine  part  rotating  at  high  speed  cracks  and  breaks  out  of  the  engine  housing.  In 
two  U.S.  accidents,  engine  parts  have  breached  the  body  of  the  aircraft  resulting  in  fatalities. 
Runway  incursions  are  occurrences  at  a  towered  or  nontowered  airport,  involving  an 
aircraft,  vehicle,  or  pedestrian  within  the  runway  safety  area,  that  creates  a  real  or  potential 
collision  hazard  with  an  aircraft  taking  off,  intending  to  take  off,  or  landing  or  intentding  to 
land. 
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fatal  accidents.  These  include,  for  example,  weather,  loss  of  control,  and 
runway  incursions.  In  establishing  a  goal  for  general  aviation,  the  initiative 
did  not  adopt  the  80-precent  goal  proposed  by  the  two  aviation  safety 
commissions.  The  initiative  chose  a  goal  of  350  fatal  general  aviation 
accidents  in  2007.  This  represents  a  20-percent  reduction  in  the  number  of 
fatal  accidents  projected  for  that  year  given  expected  growth.  The  Safer 
Skies  initiative  also  set  an  interim  goal  of  379  fatal  accidents  for  each  of  the 
next  3  years.  This  interim  goal  represents  a  7-percent  increase  over  the 
number  of  fatal  accidents  in  1999  and  does  not  challenge  the  general 
aviation  community  to  continue  the  kinds  of  safety  improvements  that 
helped  reduce  such  accidents  to  354  last  year. 

Initially,  the  Safer  Skies  initiative  focused  on  addressing  the  safety 
problems  that  caused  past  fatal  accidents  and  serious  incidents.  Aviation’s 
significant  growth  and  changes  in  the  industry’s  operations  are  likely  to 
lead  to  types  of  accidents  that  differ  from  those  in  the  past.  To  anticipate 
and  prevent  accidents  that  could  result  from  such  changes,  the  Joint  Safety 
Strategy  Initiative  in  Europe5  has  formed  a  work  group  to  develop  a  method 
for  examining  future  hazards.  Representatives  from  FAA  associated  with 
the  Safer  Skies  initiative  participate  in  this  work  group  to  help  coordinate 
the  initiative’s  work  on  past  accidents  and  incidents  with  the  international 
work  on  future  hazards.  This  work  on  future  hazards  is  preliminary  and  is 
currently  focused  on  developing  a  methodology  for  identifying  and 
addressing  these  potential  safety  problems.  As  this  work  group’s  efforts 
progress,  coordinating  these  two  efforts  should  help  avoid  duplication  of 
effort  and  foster  awareness  of  and  solutions  to  these  potential  problems 
internationally. 


The  Safer  Skies  Initiative 
Has  Made  Progress 
In  Selecting  and 
Implementing  Interventions 


As  of  April  1,  2000,  Safer  Skies  teams  had  identified  a  number  of 
interventions,  and  efforts  were  being  implemented  to  address  5  of  the  16 
safety  problems;  study  is  under  way  on  an  additional  8;  and  3  have  not  yet 
been  addressed.  The  teams  dealing  with  well-studied  safety  problems  have 
made  the  most  progress  in  selecting  and  implementing  interventions.  These 
include  uncontained  engine  failure,  controlled  flight  into  terrain,  and 
weather  in  commercial  aviation  and  controlled  flight  into  terrain  and 
weather  in  general  aviation.  Progress  occurred  because  these  teams  were 
able  to  take  advantage  of  existing  studies  and  safety  recommendations.  For 


^he  Joint  Safety  Strategy  Initiative  includes  members  from  European  aviation 
manufacturers,  associations,  and  regulators. 
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example,  the  team  reviewing  uncontained  engine  failure  has  completed  its 
work;  the  more  extensive  engine  inspections  it  recommended  are  under 
way.  In  addition,  air  carriers  had  taken  action  on  some  high-priority 
recommendations  before  FAA  issued  a  final  rule  or  the  Safer  Skies  team 
issued  its  final  report.  Specifically,  air  carriers  began  installing  the 
enhanced  navigational  equipment  in  their  aircraft  to  prevent  accidents 
from  controlled  flight  into  terrain  before  FAA  issued  its  final  rule  in  March 
2000  or  the  commercial  aviation  team  working  on  controlled  flight  into 
terrain  issued  its  final  report  in  June  2000.  Educational  information  has 
also  been  made  available  to  improve  cabin  safety  by  publicizing  the  safety 
benefits  of  greater  passenger  use  of  seat  belts  and  child  restraint  systems 
and  issues  associated  with  the  stowage  of  carry-on  baggage. 

GAO  found  that  the  process  Safer  Skies  teams  have  been  using  to  choose 
interventions  and  implement  them  is  reasonable  and  has  allowed  FAA  and 
industry  groups  to  reach  consensus  on  how  to  address  safety  problems. 
The  first  Safer  Skies  teams  that  used  this  process  were  able  to  compare 
their  results  with  those  of  prior  FAA  and  industry  studies.  The  process  for 
analyzing  data  and  selecting  interventions  has  been  modified  by  Safer  Skies 
teams  to  accommodate  differences  in  available  data  on  other  safety 
problems.  For  example,  the  runway  incursion  team  analyzed  incidents 
because  so  few  fatal  accidents  result  from  this  safety  problem.  Such 
evolution  in  the  process  will  be  critical  when  addressing  safety  problems 
that  have  not  been  studied  previously. 

The  interventions  selected  by  Safer  Skies  teams  can  reduce  the  fatal 
accident  rate  only  if  they  are  implemented  effectively.  GAO’s  past  work  has 
shown  that  FAA  does  not  consistently  follow  through  on  implementing  key 
safety  recommendations.6  The  initiative  has  developed  a  system  for 
tracking  the  implementation  of  interventions  to  improve  safety  in 
commercial  aviation.  However,  the  implementation  of  Safer  Skies’ 
interventions  is  not  assured  because  the  tracking  system  for  commercial 
aviation  is  not  sufficiently  detailed  to  assess  progress  in  implementing 
interventions.  Furthermore,  although  the  general  aviation  steering 
committee  is  approaching  final  approval  on  interventions  to  address  two 
safety  problems,  it  is  still  developing  a  tracking  system,  and  no  system  was 
developed  to  track  interventions  implemented  in  cabin  safety. 


6 Aviation  Safety:  FAA  Generally  Agrees  With  but  Is  Slow  in  Implementing  Safety 
Recommendations  (GAO/RCED-96-193,  Sept.  23, 1996). 
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The  Safer  Skies  Initiative 
Has  Yet  to  Develop  Ways  to 
Evaluate  the  Effectiveness 
of  Most  Interventions  Under 
Way 


The  Safer  Skies  initiative  has  developed  a  way  to  evaluate  the  effectiveness 
of  one  intervention  it  has  under  way  to  reduce  the  fatal  accident  rate. 
Performance  measures  are  needed  both  to  fulfill  the  mandate  of  the 
congressional  commission  that  such  performance  measures  be  established 
and  to  meet  the  requirements  of  federal  law,  which  requires  federal 
departments  requesting  funding  to  evaluate  the  effectiveness  of  their 
programs.  Since  fatal  aviation  accidents  are  infrequent,  the  effectiveness  of 
Safer  Skies’  interventions  must  be  measured  using  information  about  other 
kinds  of  events,  such  as  incidents  that  typically  precede  accidents.  The 
Safer  Skies  steering  committees  recognized  early  on  that  alternative 
measures  would  be  needed  to  measure  the  unique  effect  of  individual 
interventions.  Thus  far,  however,  only  the  uncontained  engine  failure  team 
has  developed  a  quantifiable  performance  measure.  In  contrast,  the  two 
general  aviation  teams  that  have  completed  their  work  included  no 
specific,  quantifiable  measures  for  evaluating  the  effectiveness  of  their 
interventions.  For  example,  one  team  recommended  better  marking  of 
towers  and  power  wires  to  prevent  fatal  accidents  that  result  when  low- 
flying  aircraft  strike  these  objects.  The  team’s  performance  measure  was  a 
decrease  in  the  number  of  accidents  involving  wires  and  towers.  The 
effectiveness  of  this  intervention  cannot  be  measured  without  specific, 
quantified  baseline  information  on  how  many  of  these  accidents  occurred 
in  the  past,  what  growth  is  expected  in  general  aviation,  and  what 
reduction  the  team  hopes  to  achieve  with  this  intervention.  No 
performance  measures  were  developed  to  evaluate  the  educational 
interventions  implemented  for  the  four  cabin  safety  problems. 


Coordination  Has  Been 
Extensive  but  Could  Be 
Improved  to  Enhance  the 
Impact  of  Safer  Skies’ 
Interventions 


Although  Safer  Skies  steering  committees  and  teams  included  many 
government  and  industry  aviation  experts,  three  aspects  of  coordination 
could  be  improved  to  enhance  the  chances  of  implementing  and  evaluating 
the  initiative’s  safety  interventions.  First,  the  steering  committees  for 
commercial  aviation  and  general  aviation  have  both  sought  the 
commitment  of  all  participants  to  implementing  and  funding  interventions 
before  giving  final  approval  to  move  forward.  However,  FAA’s  commitment 
has  come  at  different  points  in  the  approval  process  for  interventions 
recommended  by  these  steering  committees.  FAA’s  commitment  to  the 
general  aviation  interventions  is  still  uncertain  even  after  some  participants 
believed  that  the  steering  committee  had  granted  its  final  approval.  As  a 
consequence,  general  aviation  participants  were  more  skeptical  about 
whether  FAA  would  implement  or  fund  their  safety  interventions.  In 
October  1999,  FAA  formed  an  executive  council  to  help  coordinate  the 
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implementation  of  the  agency’s  safety  agenda,  but  this  council  has  not  yet 
documented  its  process  for  approving  and  funding  interventions. 

Second,  if  funding  is  limited,  it  remains  unclear  what  process  will  be  used 
to  reprioritize  available  resources  to  ensure  funding  for  interventions  that 
emerge  later  but  have  greater  potential  for  reducing  the  fatal  accident  rate. 
A  Safer  Skies  team  has  just  begun  work  on  loss  of  control — which  caused 
the  greatest  number  of  fatal  accidents  in  commercial  aviation  in  1988-97. 
Interventions  to  address  loss  of  control  are  thus  likely  to  be  critical  for 
reducing  the  fatal  accident  rate.  If  funding  is  limited,  some  resources  may 
have  to  be  shifted  from  existing  programs  and  safety  initiatives.  The  Safer 
Skies  initiative  and  FAA’s  executive  council  have  not  yet  established  any 
process  for  reprioritizing  interventions  if  funding  is  limited. 

Finally,  coordination  among  Safer  Skies  steering  committees,  FAA,  and 
DOT  needs  to  improve  to  ensure  the  effective  evaluation  of  Safer  Skies 
interventions.  DOT  is  responsible  for  setting  safety  goals  for  all  modes  of 
transportation  under  its  authority,  including  aviation.  Each  of  its  agencies, 
including  FAA,  provides  more  detailed  information  on  how  it  will  achieve 
those  safety  goals.  Currently,  DOT  and  FAA  measure  progress  toward  the 
goal  of  an  80-percent  reduction  in  the  fatal  accident  rate  for  commercial 
aviation  in  different  ways.  Specifically,  DOT’s  performance  plan  measures 
progress  using  a  fatal  accident  rate  based  on  flight  hours,  while  FAA’s 
strategic  plan  and  the  Safer  Skies  initiative  use  an  accident  rate  based  on 
aircraft  departures  as  the  measure  of  activity.  Since  the  ultimate  mission  of 
all  three  groups  is  to  reduce  the  fatal  accident  rate,  using  the  same  activity 
measure  to  calculate  that  rate  would  make  sense.  Because  most 
commercial  aviation  accidents  occur  during  takeoff  and  landing,  GAO 
believes  that  using  departures  better  measures  passengers’  exposure  to 
risk. 


Recommendations 


To  improve  FAA’s  safety  agenda  for  decreasing  fatal  aviation  accidents, 
GAO  makes  a  number  of  recommendations  in  this  report  to  the  Secretary 
of  Transportation  to  direct  the  FAA  Administrator  in  her  work  with  the 
Safer  Skies  steering  committees.  These  recommendations  focus  on 
establishing  a  more  challenging  goal  for  reducing  fatal  accidents  in  general 
aviation  and  improving  the  implementation  and  evaluation  of  the  Safer 
Skies  initiative. 
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Agency  Comments  and 
Our  Evaluation 


GAO  provided  copies  of  a  draft  of  this  report  to  the  Department  of 
Transportation  and  FAA  for  their  review  and  comment.  GAO  met  with  FAA 
officials,  including  the  Deputy  Associate  Administrator  for  Regulation  and 
Certification  and  the  Director  of  Aircraft  Certification.  The  FAA  officials 
concurred  with  the  majority  of  our  recommendations  and  characterized  the 
Safer  Skies  report  as  generally  “fair  and  reasonable.”  They  informed  GAO 
of  actions  taken  by  the  agency  since  GAO  completed  its  audit  work  in 
March.  This  information  has  been  incorporated  as  appropriate.  FAA 
concurred  with  the  need  to  set  more  challenging  interim  and  long-term 
goals  for  general  aviation  and  plans  to  do  so  in  the  future.  FAA  officials 
agreed  that  improvements  were  needed  in  how  the  Safer  Skies  initiative 
tracks  the  implementation  of  interventions,  although  they  disagreed  with 
the  level  of  detail  suggested  by  GAO.  They  also  agreed  with  GAO’s 
recommendations  to  improve  the  baseline  data  and  performance  measures 
used  to  evaluate  the  impact  of  these  interventions. 


The  Safer  Skies  initiative  has  taken  steps  to  expand  and  improve  its 
tracking  of  interventions,  but  the  system  still  does  not  clearly  identify  and 
specify  time  frames  for  major  commitments  and  deliverables  for  each 
approved  intervention.  Without  a  reliable  tracking  system,  FAA  and  Safer 
Skies  will  not  be  in  a  position  to  ensure  that  recommended  interventions 
are  implemented  to  improve  aviation  safety. 
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Background 


The  Federal  Aviation  Administration  (FAA)  has  forecast  continued  growth 
for  commercial  and  general  aviation  over  the  next  decade. 1  Growth  over 
the  past  few  decades  brought  innovations  to  improve  flight  safety  that 
contributed  to  a  dramatic  lowering  of  the  accident  rate  by  the  mid-1970s. 
Further  reductions  in  the  accident  rate  have,  however,  remained  elusive. 
Unless  the  current  accident  rate  can  be  reduced,  the  number  of  fatal 
accidents  is  likely  to  increase  as  aviation  operations  continue  to  grow. 
During  the  1990s,  FAA,  the  aviation  industry,  and  the  Congress  all 
acknowledged  and  studied  this  potential  danger.  They  set  ambitious  targets 
for  reducing  the  accident  rate,  made  over  a  thousand  recommendations  for 
improving  aviation  safety,  and  implemented  a  number  of  safety  initiatives. 
In  spite  of  these  efforts,  the  accident  rate,  which  is  already  low,  has 
remained  fairly  steady.  The  FAA  Administrator,  White  House  and 
congressional  task  forces,  and  aviation  industry  groups  have  concluded 
that  FAA  and  the  aviation  industry  must  coordinate  their  efforts  to 
prioritize  safety  recommendations  and  focus  resources  on  those  with  the 
most  potential  to  decrease  the  accident  rate.  In  1998,  the  FAA 
Administrator  announced  the  Safer  Skies  initiative,  a  joint  government- 
industry  effort  to  identify  and  address  the  greatest  threats  to  aviation  safety 
in  order  to  reduce  the  fatal  accident  rate  by  80  percent  by  the  year  2007. 


FAA  Expects 
Continued  Growth  in 
Aviation 


Over  the  past  several  decades,  aviation  has  grown  substantially  in  the 
United  States,  and  FAA  expects  this  growth  to  continue  into  the  next 
century.  Commercial  aviation  has  grown  consistently  since  1982,  while 
growth  in  general  aviation  has  been  less  consistent.  One  key  measure  of 
aviation  activity  shows  that  the  number  of  flight  hours  for  commercial 
aircraft  more  than  doubled  from  8  million  hours  in  1982  to  nearly  18  million 
hours  in  1999.  In  contrast,  general  aviation  activity  dropped  fairly  steadily 
from  the  early  1980s  until  1995.  While  general  aviation  has  grown  since 
1995,  it  has  not  yet  returned  to  1990  levels.  The  number  of  general  aviation 


‘Commercial  aviation  includes  both  large  air  carriers  and  commuter  air  carriers. 
Specifically,  commercial  aviation  includes  all  air  carriers  offering  scheduled  and 
nonscheduled  service  by  major  air  carriers  flying  under  14  C.F.R.  part  121  and  all  air  carriers 
operating  scheduled  service  under  14  C.F.R.  part  135.  General  aviation  aircraft  include  all 
U.S.-registered  civil  aircraft  not  operated  under  14  C.F.R.  part  121  or  part  135.  General 
aviation  includes  a  wide  variety  of  aircraft,  ranging  from  corporate  jets  to  small  piston- 
engine  aircraft  as  well  as  helicopters,  gliders,  and  aircraft  used  in  operations  such  as 
logging,  firefighting,  and  agricultural  spraying.  General  aviation  also  includes  on-demand  air 
carriers  that  operate  nonscheduled  service  under  14  C.F.R.  part  135. 
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flight  hours  decreased  by  nearly  9  percent  from  32.6  million  hours  in  1982 
to  29.9  million  hours  in  1999.  (See  fig.  1.) 


“  General  aviatjon 

—  —  Commercial  aviation 


Source:  GAO’s  analysis  of  data  from  the  National  Transportation  Safety  Board. 

FAA  has  forecast  continued  growth  for  commercial  aviation  as  well  as  for 
general  aviation  into  the  next  century.  The  number  of  planes  will  increase, 
and  these  aircraft  will  fly  more  miles,  spend  more  hours  in  the  air,  and  carry 
more  people.  For  example,  FAA  estimates  that  commercial  aviation  flight 
hours  will  grow  to  24  million  hours  in  2007— an  increase  of  37  percent  from 
1999.  In  commercial  aviation,  FAA  projects  that  the  use  of  large  air  carriers 
will  grow  at  an  annual  rate  of  4  percent,  while  the  use  of  commuter  air 
carriers  will  grow  at  3  percent  per  year.  Although  growth  has  been  more 
erratic  in  general  aviation  than  in  commercial  aviation,  FAA  projects  an 
annual  growth  rate  of  2.2  percent  for  general  aviation  into  the  next  century. 
FAA  estimates  that  general  aviation  flight  hours  will  increase  to  about  36 
million  hours  in  2007,  a  growth  of  nearly  19  percent  over  1999. 
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Fatal  Accident  Rates 
Have  Decreased  for 
U.S.  Aviation 


Even  with  the  growth  in  aviation,  fatal  accidents  remain  relatively  rare, 
especially  in  commercial  aviation.  Fatal  accident  rates  for  U.S.  aviation  are 
low  and  have  decreased  over  the  past  decades  for  both  commercial  and 
general  aviation.  The  fatal  accident  rate  can  be  calculated  as  the  number  of 
accidents  with  one  or  more  fatalities  divided  by  a  measure  of  aviation 
activity,  such  as  the  number  of  aircraft  miles  flown,  aircraft  hours  flown,  or 
departures. 


More  Fatal  Accidents  Occur 
in  General  Aviation,  but 
Commercial  Aviation 
Accidents  Can  Be 
Catastrophic 


In  the  10-year  period  preceding  the  initiative,  4,471  fatal  aviation  accidents 
occurred  in  the  United  States,  resulting  in  a  total  of  9,802  deaths.  Table  2 
shows  the  distribution  of  accidents  and  deaths  for  commercial  aviation, 
which  includes  large  and  commuter  air  carriers,  and  general  aviation, 
which  includes  on-demand  air  taxis.  General  aviation  accounted  for  the 
largest  number  of  fatal  accidents  and  deaths  in  1988-97.  The  initiative 
addresses  both  commercial  and  general  aviation,  but  increased  attention  is 
focused  on  further  improving  the  safety  of  commercial  aviation  because 
large  and  commuter  air  carriers  are  the  primary  forms  of  air  transportation 
for  most  Americans.  While  fatal  commercial  aviation  accidents  are  rare, 
large  airplane  accidents  can  cause  more  deaths  in  an  instant  than  most 
events,  other  than  wars  or  natural  disasters.  They  consequently  raise 
concerns  with  both  the  public  and  the  media,  and  commercial  aviation  is 
held  to  a  higher  standard  of  safety  than  other  forms  of  transportation.  With 
commercial  aviation  expected  to  grow  steadily  into  the  next  century, 
aviation  accidents  will  occur  with  a  frequency  that  will  be  unacceptable  to 
the  public  unless  steps  are  taken  to  decrease  the  fatal  accident  rate.  While 
such  accidents  remain  rare,  FAA  recognizes  that  the  public  demands  a  high 
standard  of  safety  and  expects  continued  improvement. 


Table  2:  Fatal  Accidents  and  Deaths  by  Type  of  Aviation  Operations,  1988-97 


Fatal  accidents  Deaths 


Type  of  operation 

Number 

Percentage 

Number 

Percentage 

Commercial  aviation 

85 

2 

1,756 

18 

General  aviation 

4,386 

98 

8,046 

82 

Total 

4,471 

100 

9,802 

100 

Source:  GAO’s  analysis  of  data  from  the  National  Transportation  Safety  Board. 
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How  Fatal  Accident  Rates 
Are  Calculated 


FAA  tracks  the  number  of  passenger  fatalities  for  various  types  of  aviation 
operations  and  calculates  accident  rates.  Basically,  the  rates  are  calculated 
by  dividing  the  number  of  accidents  with  one  or  more  fatalities  by  one  of 
the  various  measures  of  aviation  activity.  For  example,  the  fatal  accident 
rate  for  commercial  aviation  for  1988-97  is  0.058  per  100,000  flight  hours, 
which  was  calculated  by  dividing  the  number  of  fatal  accidents  (85) 2  by  the 
number  of  flight  hours  (151  million).  This  translates  into  about  one  fatal 
accident  for  every  2  million  hours  flown.  The  three  activity  measures 
generally  used  to  calculate  fatal  accident  rates  are  the  number  of  individual 
flights  (referred  to  as  departures),  aircraft  miles  flown,  and  aircraft  hours 
flown.  Each  activity  measure  reflects  different  exposures  to  the  risks 
associated  with  flying.  For  example,  most  commercial  aviation  accidents 
occur  during  takeoff  or  landing,  rather  than  during  the  cruise  phase,  which 
constitutes  the  largest  part  of  the  total  mileage  and  hours  flown.  For  this 
reason,  we  believe  that  departures  are  usually  the  best  measure  of 
exposure  to  risk. 


For  large  and  commuter  air  carriers,  all  three  fatal  accident  rates  are 
tracked.  But  for  general  aviation,  the  only  measure  of  exposure  is  the 
number  of  flight  hours  estimated  from  survey  data.  Thus,  fatal  accident 
rates  for  commercial  aviation  (large  and  commuter  air  carriers)  are 
generally  expressed  in  terms  of  the  number  of  fatal  accidents  per  100,000 
departures,  while  fatal  accident  rates  for  general  aviation  are  expressed  as 
the  number  of  fatal  accidents  per  100,000  flight  hours  as  estimated  by  FAA’s 
annual  survey.3  General  aviation  flight  hours  are  not  sufficiently  reliable  for 
use  in  calculating  a  fatal  accident  rate  for  general  aviation  because  they  are 
estimated  from  a  voluntary  survey,  according  to  FAA. 


2We  used  the  85  fatal  accidents  in  commercial  aviation  for  the  purpose  of  illustration.  When 
calculating  fetal  accident  rates  in  aviation,  the  National  Transportation  Safety  Board  (NTSB) 
excludes  accidents  that  resulted  from  sabotage  and  hijacking.  In  1988-97,  one  fatal  U.S. 
accident  resulted  from  sabotage  and  would  thus  be  excluded.  The  number  of  accidents  that 
NTSB  would  use  to  calculate  the  fatal  accident  rate  for  the  period  is  84  accidents. 

3FAA  uses  an  annual  survey  to  estimate  flight  hours  because  it  does  not  require  general 
aviation  operators  to  report  such  key  measures  as  the  number  of  hours  flown  or  the  number 
of  takeoffs.  The  General  Aviation  and  Air  Taxi  Activity  Survey  provides  FAA  with 
information  on  the  operations  of  these  aircraft. 
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Fatal  Accident  Rates  Have 
Decreased 


Over  the  past  few  decades,  the  annual  rate  of  fatal  aviation  accidents  has 
decreased  significantly  for  both  commercial  and  general  aviation.  While  the 
accident  rates  are  low,  they  have  shown  little  improvement  recently.  For 
large  commercial  air  carriers,  the  U.S.  accident  rate  was  26  fatal  accidents 
per  million  departures  in  1959.  Following  the  advent  of  large  jet  aircraft  in 
the  1960s,  the  rate  fell  to  one  or  fewer  fatal  accidents  per  million 
departures  and  has  remained  fairly  steady  for  three  decades.  The  fatal 
accident  rate  for  commuter  aircraft  has  also  fallen  over  the  last  several 


decades.  The  accident  rate  for  commuter  air  carriers  fell  from  about  2  fatal 


accidents  per  million  departures  in  1982  to  3  per  10  million  departures  in 
1996.  While  there  were  no  fatal  commuter  accidents  in  1998,  the  five  fatal 


accidents  in  1999  resulted  in  a  fatal  accident  rate  of  nine  per  million 


departures.  This  increase  in  the  fatal  accident  rate  reflects  a  1997 
narrowing  in  the  definition  of  commuter  air  carrier  to  include  only  small 
aircraft  with  nine  or  fewer  seats.4  Similarly,  the  accident  rate  for  general 
aviation  aircraft  has  dropped  since  1960.  The  fatal  accident  rate  of  six  per 
100,000  flight  hours  in  1960  fell  to  less  than  two  by  the  early  1980s.  The  fatal 
accident  rate  for  general  aviation  continued  to  decrease  fairly  steadily 
through  the  1980s,  increased  slightly  in  the  early  1990s,  and  has  dropped 
steadily  since  1995.  In  1999,  the  fatal  accident  rate  for  general  aviation  was 
1.2  fatal  accidents  per  100,000  flight  hours.  (See  fig.  2.) 


’Since  March  20, 1997,  aircraft  with  10  or  more  seats  formerly  operating  scheduled  service 
under  14  C.F.R.  part  135  have  been  required  to  follow  the  more  stringent  safety  requirements 
that  apply  to  larger  aircraft  under  14  C.F.R.  part  121. 
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Figure  2:  Fatal  Accident  Rates  for  Commercial  and  General  Aviation,  1982-99 

Per  100,000  flight  hours 
2.0 


1.5 


1.0 


0.5 


General  aviation 


Commercial  aviation 


Source:  GAO’s  analysis  of  NTSB  data. 

The  reductions  in  the  fatal  accident  rates  resulted  from  a  combination  of 
technological  advances  that  improved  safety.  In  commercial  aviation,  these 
advances  included  the  replacement  of  large,  piston-engine  aircraft  with  jet 
aircraft  with  far  more  reliable  engines,  the  development  of  navigational 
equipment  to  warn  pilots  of  impending  crashes,  better  ground  navigation 
aids,  improved  aircraft  instrumentation,  and  increased  air  traffic  radar 
coverage.  Some  of  these  improvements  have  also  benefited  smaller 
commuter  and  general  aviation  aircraft.  As  commuter  air  carriers  switched 
from  small  aircraft  to  sophisticated  turboprop  aircraft,  the  accident  rate 
among  the  larger  commuter  aircraft  became  comparable  to  that  of  large  air 
carriers. 
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If  Greater  Numbers  of 
Fatalities  Are  to  Be  Avoided, 
the  Fatal  Accident  Rate 
Must  Be  Reduced 


If  the  current  fatal  accident  rate  holds  steady  and  aviation  activity  grows  as 
FAA  has  projected,  the  increased  air  traffic  will  result  in  greater  numbers  of 
crashes  and  fatalities.  We  estimate  that  the  average  of  six  fatal  commercial 
aviation  accidents  per  year  in  1994-96  will  likely  rise  to  nine  per  year  by 
2007.  Similarly,  the  fatal  accidents  for  general  aviation  will  probably  mount 
from  an  average  of  380  in  1996-98  to  484  in  2007. 5  Table  3  shows  our 
projections  of  the  number  of  fatal  accidents  in  2007  calculated  from  FAA’s 
growth  estimates  and  the  current  fatal  accident  rate  for  each  type  of 
aviation  operation. 


Table  3:  Projected  Numbers  of  Fatal  Accidents  by  Type  of  Operation  in  2007 


Number  of  fatal  accidents 

Type  of  operation 

Annual  average8 

Projected  for  2007 

Commercial  aviation 

6 

9 

General  aviation 

380 

484 

"The  annual  average  is  for  the  baseline  years  chosen  by  FAA  and  Safer  Skies  steering 
committees  in  establishing  their  goals:  1994-96  for  commercial  aviation  and  1996-98  for 
general  aviation. 

Source:  GAO’s  analysis  of  data  from  FAA  and  NTSB. 

The  prospect  of  more  accidents  and  deaths  is  unacceptable  to  the  public, 
FAA,  and  the  aviation  industry.  Avoiding  that  outcome  means  reducing  the 
fatal  accident  rate  significantly.  The  final  report  of  the  National  Civil 
Aviation  Review  Commission  concluded  in  1997  that  the  “anticipated 
growth  in  aviation  between  now  and  the  first  quarter  of  the  next  century 
will  almost  certainly  lead  to  an  occurrence  of  aviation  accidents  with  a 
frequency  that  will  be  wholly  unacceptable  to  the  public.”  The  Commission 
called  for  a  joint  industry-government  effort  to  reduce  the  accident  rate 
substantially. 


5FAA  and  Safer  Skies  steering  committees  have  chosen  different  baseline  years  for 
commeicial  aviation  (1994-96)  and  general  aviation  (1996-98).  We  have  used  those  years  in 
table  3  projecting  the  number  of  fatal  aviation  accidents  and  in  subsequent  tables  in  this 
report  comparing  the  accident  reduction  goals  chosen  by  FAA  and  Safer  Skies  steering 
committees  with  the  80-percent  goal  set  forth  by  the  White  House  and  congressional 
commissions  on  aviation  safety. 
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FAA  and  the  Aviation 
Industry  Made  Previous 
Efforts  to  Reduce  the  Fatal 
Accident  Rate 


During  the  1990s,  FAA  and  aviation  industry  groups  had  separate  and  joint 
efforts  under  way  to  use  available  data  to  identify  and  address  the  major 
causes  of  accidents.  A  series  of  fatal  crashes  and  concern  that  the  number 
of  accidents  and  fatalities  will  increase  as  air  traffic  increases  prompted 
these  efforts  to  reduce  the  accident  rate.  Many  of  the  reports  that  resulted 
from  these  efforts  set  specific  goals  and  included  recommendations  for 
decreasing  aviation  accidents.  Although  FAA  and  the  aviation  industry 
acted  on  some  of  these  recommendations,  the  fatal  accident  rate  has 
remained  fairly  stable  but  low. 


The  effectiveness  of  previous  efforts  to  reduce  the  fatal  accident  rate  is 
believed  to  have  been  undercut  by  their  limited  scope  and  a  lack  of 
coordination  between  government  and  industry  groups.  Many  of  the 
studies  issued  during  the  1990s  were  under  the  leadership  of  either  FAA  or 
a  particular  segment  of  the  aviation  industry.  For  example,  FAA,  on  its  own, 
studied  controlled  flight  into  terrain  (CFIT)6  accidents  and  runway 
incursions.  Separately,  the  Flight  Safety  Foundation  brought  together 
participants  from  many  segments  of  the  aviation  industry  to  study  CFIT 
and  approach  and  landing  but  initially  had  only  limited  FAA  involvement. 
The  Aerospace  Industries  Association  initiated  an  extensive  study  on  the 
causes  of  safety-related  problems  in  aircraft  engines,  including 
uncontained  engine  failure.7  (For  a  list  of  key  aviation  studies  and  our 
related  reports,  see  app.  II.)  According  to  FAA  and  industry  officials  we 
interviewed,  efforts  to  address  specific  safety  issues  were  generally 
unsuccessful  when  one  group  failed  to  coordinate  its  work  with  that  of 
other  groups  that  had  important  roles  in  aviation  safety. 

Many  of  these  reports  issued  during  the  1990s  set  specific  goals  for 
reducing  the  overall  fatal  accident  rate  or  for  addressing  specific  aviation 
safety  problems  that  result  most  often  in  fatalities.  They  also  included 
numerous  specific  recommendations  to  FAA  and  the  aviation  industry  to 
help  meet  these  goals.  Among  the  key  reports  were  the  following: 


6CFIT  is  flying  an  otherwise  controllable  aircraft  into  the  ground  or  water. 

7Uncontained  engine  failure  occurs  when  a  heavy  engine  part  rotating  at  high  speed  cracks 
and  breaks  out  of  the  engine  housing.  In  two  U.S.  accidents,  engine  parts  breached  the  body 
of  the  aircraft.  One  accident  on  takeoff  resulted  in  the  death  of  2  passengers,  while  the  other 
accident  crippled  key  aircraft  systems  in  flight,  resulting  ultimately  in  a  crash  that  killed  1 1 1 
passengers  and  crew. 
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•  In  1993,  the  Flight  Safety  Foundation  led  an  international  task  force  on 
CFIT,  the  leading  cause  of  fatal  commercial  aviation  accidents 
worldwide.  The  task  force  provided  specific  recommendations  and 
training  aids  aimed  at  reducing  CFIT  accidents.  The  task  force  set  a  goal 
of  reducing  these  accidents  50  percent  worldwide  by  1998  and  other 
goals  targeting  improvements  in  the  regions  of  the  world  with  the 
highest  CFIT  rates.8 

•  In  January  1995,  over  1,000  government,  industry,  and  union  officials 
attended  an  FAA-sponsored  safety  conference.  The  officials  agreed  that 
they  shared  responsibility  for  pursuing  a  goal  of  zero  accidents.  Their 
report  identified  173  high-priority  safety  initiatives  in  the  areas  of  crew 
training,  air  traffic  control  and  weather,  safety  data  collection  and  use, 
applications  of  emerging  technologies,  aircraft  maintenance  procedures 
and  inspections,  and  development  of  flight  operating  procedures.9 

•  Following  the  May  1996  Valujet  crash,  an  FAA  task  force  recommended 
in  September  1996  that  FAA  target  agency  resources  to  safety  risks, 
improve  the  certification  and  oversight  of  new  air  carriers,  and  address 
concerns  about  inspector  guidance  and  resources.10 

•  In  February  1997,  the  White  House  Commission  on  Aviation  Safety  and 
Security  recommended  that  the  government  and  the  aviation  industry 
establish  a  national  goal  to  reduce  the  aviation  fatal  accident  rate  by  a 
factor  of  five  (meaning  80  percent)  within  10  years.  To  achieve  that  goal, 
the  Commission  made  specific  recommendations  for  reengineering 
FAA’s  regulatory  and  certification  programs.11  The  Commission  did  not 
explicitly  state  whether  the  national  goal  should  apply  to  all  types  of 
aviation  operations. 


’Flight  Safety  Foundation.  Controlled  Flight  Into  Terrain:  Education  and  Training  Aid 
(Disseminated  under  the  sponsorship  of  the  U.S.  Department  of  Transportation.). 

’Zero  Accidents  ...A  Shared  Responsibility,  U.S.  Dept,  of  Transportation,  FAA,  Feb.  9, 1995. 

mFAA  90  Day  Safety  Review,  U.S.  Dept,  of  Transportation,  FAA,  Sept.  16, 1996. 

"Final  Report  to  President  Clinton,  White  House  Commission  on  Aviation  Safety  and 
Security,  Feb.  12,  1997. 
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•  In  December  1997,  the  National  Civil  Aviation  Review  Commission 
recommended  that  the  government  and  the  aviation  industry  work 
together  to  achieve  the  White  House  Commission’s  goal  of  an  80-percent 
reduction  in  the  accident  rate  over  the  next  10  years  and  recommended 
specific  safety  improvements  for  achieving  that  goal.12  While  the 
Commission  did  not  explicitly  state  whether  the  80-percent  goal  should 
apply  to  all  types  of  aviation  operations,  the  Commission  specifically 
discussed  the  accident  rates  for  large  jets,  commuter  air  carriers, 
general  aviation  operations,  and  air  taxis. 

Both  the  White  House  Commission  on  Aviation  Safety  and  Security  and  the 
National  Civil  Aviation  Review  Commission  called  for  FAA  and  the  aviation 
industry  to  work  together  on  aviation  safety  issues. 


The  Safer  Skies 
Initiative  Continued 
Ongoing  Efforts  to  Use 
Data  Analysis  to 
Address  Safety 
Problems 


On  April  14, 1998,  the  Vice  President,  the  Secretary  of  Transportation,  and 
the  FAA  Administrator  announced  the  Safer  Skies  initiative,  a  new  aviation 
safety  program  committed  to  reducing  the  fatal  accident  rate  by  80  percent 
by  2007.  Under  the  initiative,  experts  from  FAA,  the  aviation  industry,  and 
other  government  agencies  with  responsibility  for  aviation  are  to  jointly 
analyze  U.S.  and  global  data  to  identify  the  most  serious  threats  to  aviation 
and  to  find  the  root  causes  of  accidents.  They  will  then  determine  the  best 
actions  to  break  the  chain  of  events  that  lead  to  accidents  and  direct 
resources  first  to  those  actions.  These  actions  are  also  referred  to  as 
interventions. 


FAA  Invited  Members  of 
Ongoing  Industry  and 
Government  Safety  Groups 
to  Join  the  Safer  Skies 
Initiative 


When  FAA  announced  the  Safer  Skies  initiative,  the  agency  invited 
participants  from  a  number  of  ongoing  industry  and  government  safety 
groups  to  join  in  creating  a  unified  safety  agenda.  In  establishing  the 
agenda  for  the  initiative,  the  commercial  and  general  aviation  steering 
committees  joined  with  and  expanded  the  preexisting  efforts.  To  develop 
the  unified  agenda,  key  government  and  industry  aviation  officials  are  to 
conduct  data  analyses  to  identify  the  causes  of  fatal  accidents  and 
determine  what  interventions  are  needed  to  prevent  them. 


12  Avoiding  Aviation  Gridlock  and  Reducing  the  Accident  Rate:  A  Consensus  for  Change, 
National  Civil  Aviation  Review  Commission,  Dec.  11, 1997. 
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Several  of  the  preexisting  safety  groups  were  already  using  data-driven 
approaches  to  study  aviation  safety  issues.  Representatives  of  air  carriers, 
aircraft  and  engine  manufacturers,  and  related  associations  had 
established  a  commercial  aviation  group  in  January  1997  to  analyze  fatal 
commercial  aviation  accidents  and  to  recommend  ways  to  prevent  them. 
Before  joining  the  initiative,  this  group  had  outlined  a  process  for  obtaining 
accident  data  from  U.S.  and  international  sources  and  for  reaching 
consensus  on  the  safety  problems  to  be  addressed.  Another  industry  group 
analyzing  data  on  uncontained  engine  failure  had  developed  a  process  for 
analyzing  safety  data,  using  case  studies  to  identify  root  causes,  and 
evaluating  the  feasibility  of  proposed  interventions.  A  third  group  that 
represented  a  cross-section  of  various  general  aviation  constituencies, 
such  as  pilots  and  small  aircraft  manufacturers,  was  addressing  the  causes 
of  fatal  general  aviation  accidents.13  A  joint  government-industry  group 
sponsored  by  FAA  was  continuing  work  on  issues  pertaining  to  the  safety 
of  passengers  and  crew  members  in  the  aircraft  cabin  that  had  been  started 
separately  by  FAA,  industry  associations,  and  unions  representing  flight 
attendants.14  FAA  invited  members  from  all  four  of  these  groups  to 
participate  in  the  initiative. 


Steering  Committees 
Selected  16  Safety 
Problems  for  the  Safer  Skies 
Initiative  to  Address 


Safer  Skies  formed  steering  committees  of  safety  experts  from  government 
and  industry  to  lead  the  work  in  each  of  its  three  agenda  areas:  commercial 
aviation,  general  aviation,  and  cabin  safety.  Each  steering  committee  has 
co-chairs  and  participants  from  both  FAA  and  industry.  The  commercial 
and  general  aviation  steering  committees  used  available  data  to  select  the 
safety  problems  to  be  addressed  in  their  respective  agenda  areas.  In 
contrast,  the  cabin  safety  steering  committee  continued  the  work  on  safety 
problems  that  had  already  been  under  way  as  a  joint  FAA-industry  effort 
that  preceded  Safer  Skies.  The  three  Safer  Skies  steering  committees 
ultimately  chose  to  address  16  safety  problems:  6  in  commercial  aviation,  6 
in  general  aviation,  and  4  in  cabin  safety.  The  commercial  aviation  and 
general  aviation  steering  committees  selected  several  of  the  same  safety 
problems,  including  weather  and  loss  of  control  over  the  aircraft.  Because 
safety  problems  can  affect  large  and  small  aircraft  differently,  the 
commercial  and  general  aviation  steering  committees  planned  to  have 


13Prior  to  joining  the  Safer  Skies  initiative,  the  Joint  Safety  Coalition  was  known  as  the 
General  Aviation  Coalition. 

14This  gi  oup  was  called  Partners  in  Cabin  Safety. 
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separate  teams  study  each  safety  problem  with  one  exception.  A  joint  team 
will  study  runway  incursions  because  commercial  and  general  aviation 
aircraft  often  share  the  same  runways  and  accidents  have  occurred 
involving  both  types  of  aircraft.  Table  4  lists  and  briefly  explains  each 
safety  problem. 

Table  4:  The  16  Safety  Problems  Addressed  by  Safer  Skies 

Problem  identified 

Definition  of  problem 

Commercial  aviation 

Controlled  flight  into  terrain 

Flying  an  otherwise  controllable  aircraft  into  terrain. 

Loss  of  control 

Accidents  resulting  from  situations  in  which  the  pilot  should  have  maintained  or 
regained  aircraft  control  but  did  not. 

Uncontained  engine  failure 

Small  cracks  in  high-speed  rotating  parts  that,  if  left  undetected,  can  propagate  until 
failure. 

Runway  incursion 

An  occurrence  at  a  towered  or  nontowered  airport,  involving  an  aircraft,  vehicle,  or 
pedestrian  within  the  runway  safety  area,  that  creates  a  real  or  potential  collision 
hazard  with  an  aircraft  taking  off,  intending  to  take  off,  or  landing  or  intending  to  land. 

Approach  and  landing 

Situations  in  which  a  pilot  may  have  been  able  to  land  an  aircraft  safely,  but  did  not. 

Weather 

Meteorological  conditions  (icing,  turbulence,  etc.)  that  adversely  affect  aircraft 
performance. 

General  aviation 

Controlled  flight  into  terrain 

Flying  an  otherwise  controllable  aircraft  into  terrain. 

Loss  of  control 

Accidents  resulting  from  situations  in  which  the  pilot  should  have  maintained  or 
regained  aircraft  control  but  did  not. 

Runway  incursion 

An  occurrence  at  a  towered  or  nontowered  airport,  involving  an  aircraft,  vehicle,  or 
pedestrian  within  the  runway  safety  area,  that  creates  a  real  or  potential  collision 
hazard  with  an  aircraft  taking  off,  intending  to  take  off,  or  landing  or  intending  to  land. 

Aeronautical  decision-making 

Often  a  fundamental  element  in  accident  causal  chains,  where  a  pilot  did  not  make  the 
best  safety  decision  about  a  flying  or  nonflying  situation. 

Weather 

Meteorological  conditions  (icing,  turbulence,  etc.)  that  adversely  affect  aircraft 
performance. 

Survivability 

Safety  research  and  initiatives  that  would  reduce  fatalities. 

Cabin  safety 

Passenger  interference 

Passengers  who  hinder  crew  members  in  performing  their  duties. 

Passenger  seat  belt  use 

Injuries  sustained  by  passengers  who  are  not  wearing  their  seat  belts  when 
encountering  unexpected  air  turbulence. 

Carry-on  baggage 

Articles  brought  into  the  airplane  cabin  by  passengers. 

Child  safety  restraints 

Safety  issues  associated  with  the  commercial  aviation  transportation  of  children  under 
the  age  of  2  years  old. 

Source:  FAA. 
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Objectives,  Scope,  and 
Methodology 


At  the  request  of  the  Chairman  and  Ranking  Democratic  Member  of  the 
Subcommittee  on  Aviation,  House  Committee  on  Transportation  and 
Infrastructure,  we  reviewed  the  design  and  implementation  of  the  Safer 
Skies  initiative.  Specifically,  they  asked  us  to  determine  (1)  to  what  extent 
addressing  the  safety  problems  selected  by  the  Safer  Skies  initiative  will 
help  reduce  the  fatal  accident  rate;  (2)  what  progress  the  initiative  has 
made  in  identifying  and  implementing  interventions  to  address  each  of 
these  safety  problems;  (3)  what  progress  the  Safer  Skies  initiative  has  made 
in  assessing  the  effectiveness  of  those  interventions;  and  (4)  how  FAA  is 
coordinating  the  Safer  Skies  initiative  with  other  safety  activities 
conducted  throughout  the  agency,  in  partnership  with  the  aviation  industry, 
and  by  other  federal  agencies. 


Because  Safer  Skies  is  a  10-year  project  that  hopes  to  reach  its  goals  in 
2007,  we  analyzed  domestic  flight  operations  and  accident  data  for  the 
decade  that  preceded  the  1998  announcement  of  Safer  Skies  and  for  the 
decade  to  come.  We  examined  data  on  fatal  accidents  and  their  causes  for 
all  types  of  aviation  operations  in  the  United  States  from  1988  through 
1997.  We  also  examined  projected  data  for  aviation  operations  and 
accidents  through  2007.  » 


To  determine  whether  addressing  the  safety  problems  chosen  by  Safer 
Skies  will  help  reduce  the  fatal  accident  rate,  we  interviewed  FAA  officials 
responsible  for  overseeing  Safer  Skies,  officials  at  the  Department  of 
Defense  and  the  National  Aeronautics  and  Space  Administration  involved 
in  aviation  safety,  and  the  chairs  and  many  members  of  the  steering 
committees  for  commercial  aviation,  general  aviation,  and  cabin  safety.  We 
reviewed  documents  related  to  each  of  these  steering  committees  as  well 
as  data  used  by  these  groups  in  choosing  the  problems  on  which  Safer 
Skies  would  focus.  We  also  discussed  the  problems  Safer  Skies  selected  as 
priorities  with  staff  at  the  National  Transportation  Safety  Board,  the  Flight 
Safety  Foundation,  and  other  aviation  safety  groups. 

To  determine  what  progress  has  been  made  in  identifying,  developing,  and 
implementing  intervention  strategies  for  the  Safer  Skies  initiative,  we 
interviewed  the  FAA  and  industry  chairs  of  the  teams  formed  to  address 
each  problem  under  study.  We  obtained  and  reviewed  team  reports 
completed  for  each  safety  problem  to  understand  the  analysis  process, 
modifications  made  to  it  by  successive  work  groups,  and  actions  planned 
to  improve  aviation  safety  in  each  problem  area. 
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To  determine  what  progress  has  been  made  to  date  in  assessing  the 
effectiveness  of  its  actions  to  improve  aviation  safety,  we  reviewed 
implementation  plans  to  determine  whether  schedules  were  being  met  and 
whether  ways  had  been  chosen  to  measure  the  success  of  such  actions.  We 
also  reviewed  available  team  reports  and  relevant  data  to  determine 
whether  sufficient  data  were  available  to  measure  Safer  Skies’  progress  in 
improving  aviation  safety. 

To  determine  how  FAA  coordinated  the  Safer  Skies  initiative  with  safety 
activities  conducted  throughout  FAA  and  in  partnership  with  the  aviation 
industry,  we  reviewed  information  on  related  industry  and  government 
safety  activities.  Specifically,  we  sought  information  on  activities  under  the 
auspices  of  FAA,  the  Department  of  Defense,  the  National  Aeronautics  and 
Space  Administration,  the  National  Transportation  Safety  Board,  selected 
engine  and  aircraft  manufacturers,  several  major  air  carriers,  the  Air 
Transport  Association,  and  the  Aircraft  Owners  and  Pilots  Association. 
During  our  interviews  with  members  of  the  Safer  Skies  steering 
committees  and  teams,  we  discussed  efforts  to  coordinate  their  work  with 
other  government  and  industry  safety  activities.  We  also  reviewed  the 
reports  from  each  Safer  Skies  team  for  safety  problems  where  coordination 
would  be  appropriate.  We  discussed  the  budgetary  implications  of  the  Safer 
Skies  initiatives  and  the  criteria  for  prioritizing  resources  with  FAA  officials 
and  steering  committee  members. 

We  conducted  our  work  from  August  1999  through  June  2000  in  accordance 
with  generally  accepted  government  auditing  standards. 
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The  Safer  Skies  Initiative  Should  Help 
Improve  Aviation  Safety 


Addressing  the  16  safety  problems  chosen  by  the  Safer  Skies  initiative 
should  help  reduce  the  nation’s  fatal  accident  rate.  In  commercial  aviation, 
eliminating  the  six  safety  problems  to  be  addressed  by  the  initiative  would 
approach  the  80-percent  goal.  Other  FAA  initiatives  are  addressing 
additional  safety  problems  in  commercial  aviation,  which  should 
complement  efforts  under  the  Safer  Skies  initiative.  In  general  aviation,  the 
initiative  will  address  six  problems  that  appear  to  be  among  the  most 
common  causes  of  fatal  accidents  for  this  type  of  operation,  according  to 
available  accident  data.  While  the  initiative  has  adopted  the  80-percent  goal 
in  commercial  aviation,  which  transports  most  passengers  who  fly  in  the 
United  States,  the  initiative  adopted  a  less  aggressive  goal  for  general 
aviation,  which  accounted  for  the  vast  majority  of  the  fatal  aviation 
accidents.  The  goal  in  general  aviation  is  to  reduce  the  number  of  fatal 
accidents  to  350  in  2007,  which  represents  about  a  20-percent  reduction. 
Finally,  the  initiative  addressed  four  problems  in  cabin  safety.  Improving 
cabin  safety  will  have  little  impact  on  lowering  the  fatal  accident  rate 
because  cabin  safety  accounted  for  only  two  U.S.  fatalities  in  commercial 
aviation  in  1988-97.  No  quantitative  goal  was  set  for  safety  improvements  in 
cabin  safety.  To  date,  safety  improvement  efforts  by  FAA  and  the  Safer 
Skies  initiative  have  focused  on  past  accidents  and  incidents,  which  may 
not  be  entirely  predictive  of  future  ones.  Studying  growth  and 
technological  changes  in  the  aviation  industry  can  help  anticipate  and 
prevent  the  safety  problems  and  accidents  that  are  likely  to  arise  from  such 
changes.  Preliminary  international  efforts  have  been  initiated  to  address 
future  hazards,  and  coordinating  these  efforts  with  Safer  Skies  work  could 
enhance  the  initiatives  efforts  to  reduce  the  fatal  accident  rate. 


The  Safer  Skies 
Initiative  Addresses 
Major  Safety  Problems 
in  Commercial  Aviation 


The  Safer  Skies  initiative  plans  to  address  she  safety  problems  that 
accounted  for  79  percent  of  the  fatal  accidents  in  commercial  aviation  in 
1988-97.  If  past  accident  causes  continue,  completely  eliminating  these  six 
safety  problems  might  approach  the  80-percent  goal.  FAA  also  has  safety 
initiatives  under  way  to  address  several  of  the  safety  problems  in 
commercial  aviation  not  addressed  by  the  initiative.  These  include 
sabotage,  fuel  tank  explosions,  and  structural  problems.  In  combination 
with  the  Safer  Skies  initiative,  FAA’s  safety  initiatives  have  potential  for 
reducing  the  fatal  accident  rate  in  commercial  aviation.  For  commercial 
aviation,  the  Safer  Skies  initiative  has  established  a  goal  of  reducing  the 
fatal  accident  rate  by  80  percent  in  2007  in  accordance  with  the  goal 
envisioned  by  the  White  House  and  congressional  commissions  on  aviation 
safety. 
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The  Safer  Skies  Initiative 
Identified  Six  Major  Safety 
Problems 


The  Safer  Skies  initiative  will  address  six  safety  problems  that  accounted 
for  79  percent  of  the  fatal  commercial  aviation  accidents  in  1988-97.  Three 
of  these  safety  problems  were  major  ones  both  worldwide  and  in  the 
United  States:  pilots’  losing  control  of  their  aircraft,  pilots’  flying  otherwise 
controllable  aircraft  into  the  ground  or  water  (CFIT),  and  accidents  during 
approach  and  landing.  These  three  safety  problems  accounted  for  58  of  the 
85  fatal  accidents  in  U.S.  commercial  aviation  during  this  period.  The 
commercial  aviation  teams  are  examining  34  of  these  accidents,  which 
involved  larger  aircraft.  The  commercial  aviation  steering  committee 
referred  the  remaining  24  fatal  accidents  to  the  general  aviation  steering 
committee  for  review  because  they  involved  small  commuter  aircraft  with 
nine  or  fewer  seats  that  operated  scheduled  commercial  service.  This  was 
done  because  (1)  the  aircraft  involved  are  more  similar  to  general  aviation 
aircraft  than  to  larger  commercial  aircraft,  (2)  the  types  of  operating 
environments  and  safety  problems  that  caused  the  accidents  more  closely 
resemble  those  of  general  aviation  than  those  of  commercial  aviation,  and 
(3)  the  interventions  to  address  safety  problems  in  general  aviation  are 
more  likely  to  correct  these  safety  problems  than  interventions  designed 
for  large  commercial  aircraft.  We  reviewed  the  National  Transportation 
Safety  Board’s  (NTSB)  reports  for  the  24  small  commuter  accidents  and 
found  that  most  of  the  accidents  happened  in  Alaska  when  pilots  flew  into 
mountains  after  deteriorating  weather  reduced  visibility.  On  the  basis  of 
our  review,  we  concur  with  the  commercial  aviation  steering  committee’s 
assessment  that  these  accidents  more  closely  resemble  general  aviation 
accidents  and  would  likely  benefit  from  the  interventions  that  emerge  to 
address  these  safety  problems  in  general  aviation  aircraft. 
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The  potential  for  improving  safety  in  these  smaller  commuter  aircraft 
exists  with  a  number  of  the  interventions  proposed  by  the  general  aviation 
teams  working  on  weather  and  CFIT.  It  is  unclear  whether  the  initiative  or 
FAA  has  mechanisms  in  place  to  ensure  that  small  commuter  operators  will 
benefit  from  the  interventions  developed.  For  example,  many  of  the 
interventions  involve  providing  additional  training  to  pilots  on  weather 
conditions  and  assessing  the  risk  factors  associated  with  each  flight. 
Because  the  initiative  plans  to  deliver  much  of  this  training  jointly  with  the 
Aircraft  Owners  and  Pilots  Association,  it  is  essential  that  notification 
about  this  training  also  be  provided  to  small  commuter  operators  and  pilots 
who  could  benefit  from  this  training  but  may  not  be  members.  Although 
members  of  several  other  organizations  participate  in  the  general  aviation 
steering  committee  and  study  teams,1  neither  the  initiative  nor  FAA  has 
made  specific  provisions  to  ensure  that  such  interventions  are  also  directed 
at  small  commuter  aircraft  operators  and  pilots,  as  well  as  at  general 
aviation  pilots.  Because  small  commuter  accidents  accounted  for  28 
percent  of  the  85  fatal  accidents  in  commercial  aviation  in  1988-97, 
reducing  the  fatal  accident  rate  by  80  percent  depends  on  addressing  these 
safety  problems  in  small  commuter  aircraft,  as  well  as  in  large  commercial 
aircraft. 

To  further  reduce  the  fatal  accident  rate  for  commercial  aviation,  the 
initiative  will  address  three  additional  safety  problems  that  have  resulted  in 
fewer  fatal  accidents  in  the  United  States  from  1988  through  1997.  The 
steering  committee  chose  runway  incursions,  uncontained  engine  failure, 
and  weather,  each  of  which  resulted  in  from  two  to  four  fatal  accidents. 
These  three  safety  problems  accounted  for  an  additional  9  accidents,  or  1 1 
percent  of  the  85  fatal  accidents.  The  committee  selected  these  problems 
because  they  caused  past  fatal  accidents  or  serious  incidents  that  could 
have  cost  many  lives.  These  areas  were  also  included  because  each 
occurred  with  greater  frequency  in  the  United  States  than  worldwide  and 
because  FAA  or  the  aviation  industry  had  already  begun  work  on  these 
safety  problems.  (See  fig.  3.) 


'Other  participating  groups  include,  for  example,  National  Air  Transportation  Association, 
the  Helicopter  Association  International,  and  the  National  Business  Aviation  Association. 
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Figure  3:  Safety  Problems  That  Caused  Fatal  Accidents  in  U.S.  Commercial  Aviation, 
1988-97 
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Commercial  aviation  safety  problems  handled  by  the  Safer  Skies  commercial  aviation  team 


Commercial  aviation  safety  problems  referred  to  the  general  aviation  steering  committee 


Commercial  aviation  safety  problems  not  addressed  by  Safer  Skies 


Source:  FAA. 

Our  analysis  of  aviation  data  and  review  of  safety  reports  confirmed  that 
the  initiative  is  addressing  three  major  safety  problems  that  caused  fatal 
accidents  in  commercial  aviation,  as  well  three  other  safety  problems  that 
have  the  potential  to  cause  accidents  with  large  numbers  of  fatalities. 
Reducing  or  eliminating  safety  problems  resulting  from  CFIT,  loss  of 
control,  approach  and  landing,  runway  incursions,  weather,  and 
uncontained  engine  failure  should  help  lower  the  fatal  accident  rate.  Safer 
Skies  participants,  FAA  officials,  and  industry  aviation  experts  whom  we 
interviewed  also  believe  that  the  initiative  is  addressing  the  most  important 
aviation  safety  problems.  Most  of  these  aviation  experts  indicated  strong 
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support  for  addressing  such  major  safety  concerns  as  CFIT,  approach  and 
landing,  and  loss  of  control.  Furthermore,  because  of  the  increasingly 
global  nature  of  commercial  aviation,  addressing  these  safety  problems 
means  that  many  of  the  interventions  recommended  by  the  initiative  might 
have  applicability  worldwide,  as  well  as  in  the  United  States.  Many  aviation 
experts  we  interviewed  also  supported  the  inclusion  of  safety  problems 
with  fewer  fatalities  but  with  a  high  potential  for  fatalities,  such  as  runway 
incursions  and  uncontained  engine  failure.  They  agreed  that  reducing  or 
eliminating  these  safety  problems  should  help  reduce  the  fatal  accident 
rate. 


Addressing  Additional 
Safety  Problems  Could 
Further  Reduce  the  Fatal 
Accident  Rate 


In  addition  to  successfully  addressing  the  major  safety  problems  discussed 
above,  addressing  additional  safety  problems  could  further  reduce  the  fatal 
accident  rate  in  commercial  aviation.  FAA  has  a  number  of  aviation  safety 
initiatives  under  way  that  potentially  can  contribute  to  improvements  in  the 
safety  of  smaller  commuter  aircraft  sometimes  used  in  commercial 
aviation.  For  example,  FAA’s  Capstone  Project  focuses  on  improving 
general  aviation  safety  in  Alaska  by  providing  additional  navigational  aids 
but  also  has  potential  application  for  addressing  the  safety  problems  of 
small  commuter  aircraft  used  elsewhere.  FAA  also  has  ongoing  initiatives 
to  address  the  causes  of  4  of  the  18  commercial  aviation  accidents  not 
being  addressed  by  Safer  Skies  teams.  These  include  programs  overseen  by 
the  agency’s  Office  of  Civil  Aviation  Security  to  reduce  the  threat  of 
sabotage,  hijacking,  and  the  transportation  of  hazardous  cargo.  Other  FAA 
initiatives  are  under  way  to  address  the  structural  problems  of  aging 
aircraft  and  fuel  tank  explosions.  FAA  has,  for  example,  published  a  notice 
proposing  requirements  for  design  reviews  and  mandatory  maintenance 
actions  for  fuel  tank  systems  on  large  transport  aircraft. 

Of  the  remaining  fatal  accidents  in  commercial  aviation  that  the  initiative  is 
not  addressing,  12  were  on-ground  fatalities,  and  2  resulted  from  other 
causes.2  The  on-ground  accidents  each  involved  the  death  of  a  single 
worker  or  unauthorized  individual  at  the  airport.  Most  of  these  accidents 
occurred  near  the  boarding  gate  or  ramp.  For  example,  several  employees 
were  fatally  injured  when  struck  by  an  aircraft’s  propeller  or  nose  gear 


2In  one  of  these  two  accidents,  a  private  general  aviation  aircraft  collided  in  flight  with  a 
commuter  aircraft,  which  landed  safely.  The  other  accident  involved  the  on-board  fatality  of 
a  pilot  who  entered  an  unpressurized  area  of  a  cargo  aircraft.  The  co-pilot  landed  the 
aircraft  without  further  incident. 
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during  the  course  of  their  work.  Of  the  on-ground  fatalities,  two  resulted 
from  individuals  gaining  unauthorized  access  to  airport  areas  that  should 
have  been  secured,  nine  involved  various  airline  or  airport  employees  who 
sustained  injuries  in  the  workplace,  and  one  involved  a  passenger  who  fell 
out  of  an  aircraft  catering  door  and  onto  the  ground.  Because  on-ground 
accidents  accounted  for  14  percent  of  the  85  fatal  accidents  in  commercial 
aviation  in  1988-97,  reducing  the  fatal  accident  rate  by  80  percent  by  2007 
will  be  difficult  if  these  safety  problems  are  not  addressed. 

FAA  has  initiatives  to  address  some  of  the  safety  problems  that  caused  on¬ 
ground  fatalities,  but  it  is  unclear  how  systematically  these  problems  are 
being  addressed.  Specifically,  FAA’s  Office  of  Civil  Aviation  Security 
oversees  airline  and  airport  programs  to  limit  access  to  secure  areas  to 
authorized  individuals.  The  status  of  FAA’s  efforts  to  address  workplace 
safety  issues  that  resulted  in  on-ground  fatalities  is  less  clear.  FAA  is 
responsible  for  regulating  the  safety  and  health  aspects  of  the  work 
environment  of  aircraft  crew  members  when  the  aircraft  is  in  operation. 
However,  FAA  has  not  promulgated  specific  regulations  that  address  all 
employee  safety  and  health  issues  associated  with  working  conditions  on 
aircraft.  FAA  held  a  public  meeting  in  December  1999  to  gather  information 
on  issues  associated  with  working  conditions  on  and  around  aircraft  and  to 
determine  whether  additional  regulations  should  be  proposed.  However, 
FAA  does  not  currently  have  a  group  addressing  workplace  safety  issues 
and  could  not  identify  any  regulations,  guidance,  or  other  initiatives  that 
have  been  developed  to  address  the  types  of  workplace  safety  problems 
that  caused  most  of  the  on-ground  fatalities. 


Improving  Commercial 
Aviation  Safety  Involves 
Considering  More  Than 
Reducing  the  Fatal  Accident 
Rate 


Looking  at  the  number  of  fatalities  associated  with  various  safety 
problems,  as  well  as  their  contribution  to  the  fatal  accident  rate,  provides 
additional  perspective  on  Safer  Skies’  commercial  aviation  agenda. 
Reductions  in  the  fatal  accident  rate  are  closely  linked  to  reductions  in  the 
number  of  fatal  accidents.  Following  this  logic,  the  greatest  reductions  in 
the  fatal  accident  rate  can  be  achieved  by  eliminating  the  safety  problems 
that  caused  the  greatest  number  of  accidents  with  one  or  more  fatalities. 
However,  strict  adherence  to  the  goal  of  reducing  the  fatal  accident  rate 
could  result  in  focusing  attention  and  resources  on  the  causes  of  accidents 
that  resulted  in  single  fatalities,  rather  than  on  those  causes  that  result  in 
multiple  fatalities,  as  well  as  multiple  accidents.  In  choosing  which  safety 
problems  to  address,  the  commercial  aviation  steering  committee  selected 
safety  problems  that  will  help  reduce  fatalities,  as  well  as  the  fatal  accident 
rate. 
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The  fatal  accident  rate  in  commercial  aviation  can  most  quickly  be  reduced 
by  addressing  the  three  safety  problems  that  form  the  core  of  the  Safer 
Skies  agenda  in  commercial  aviation:  CFIT,  loss  of  control,  and  approach 
and  landing.  These  three  safety  problems  accounted  for  34  fatal  accidents 
involving  larger  aircraft  that  commercial  aviation  teams  are  handling  and 
24  additional  small  commuter  accidents  that  general  aviation  teams  are 
handling.  (See  table  5.)  If  the  initiative  is  successful  in  developing  and 
implementing  interventions  to  eliminate  these  three  safety  problems  for 
both  large  aircraft  and  small  commuter  aircraft,  it  would  make  progress 
toward  preventing  the  kinds  of  safety  problems  that  caused  68  percent  of 
the  fatal  accidents  in  1988-97.  If  the  initiative  could  successfully  eliminate 
all  six  safety  problems  on  its  agenda  for  commercial  aviation,  it  would 
approach  the  goal  of  an  80-percent  reduction  in  the  fatal  accident  rate. 
However,  other  safety  problems  actually  resulted  in  more  fatal  accidents 
and  thus  could  reduce  the  fatal  accident  rate  more  quickly  if  eliminated. 
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Table  5:  Numbers  of  Fatal  Accidents  and  Fatalities  in  Commercial  Aviation  1988-97 

Fatal  accidents 

Fatalities 

Number 

Percentage8 

Number 

Percentage8 

Commercial  aviation  accidents  to  be 
addressed  by  Safer  Skies  commercial 
aviation  teams 

Loss  of  control 

19 

22.4 

530 

30.2 

Approach  and  landing 

9 

10.6 

128 

7.3 

CFIT 

6 

7.1 

339 

19.3 

Runway  incursion 

4 

4.7 

25 

1.4 

Weather 

3 

3.5 

16 

0.9 

Uncontained  engine  failure 

2 

2.4 

113 

6.4 

(Subtotal) 

43 

50.6 

1,151 

65.5 

Commercial  aviation  accidents  involving 
small  commuter  aircraft  to  be  addressed 
by  Safer  Skies  general  aviation  teams 

CFIT 

16 

18.8 

38 

2.2 

Loss  of  control 

7 

8.2 

37 

‘  2.1 

Approach  and  landing 

1 

1.2 

5 

0.3 

(Subtotal) 

24 

28.2 

80 

4.6 

Commercial  aviation  accidents  to  be 
addressed  by  other  FAA  safety  initiatives 

Structural  problems 

2 

2.4 

10 

0.6 

Fuel  tank  explosion 

1 

1.2 

230 

13.1 

Sabotage 

1 

1.2 

270 

15.4 

(subtotal) 

4 

4.8 

510 

29.1 

Gate/ramp 

12 

14.1 

12 

0.7 

Other 

2 

2.4 

3 

0.2 

Total 

85 

100 

1,756 

100 

aTotals  do  not  add  to  100  because  of  rounding. 

Source:  GAO's  analysis  of  data  from  FAA  and  NTSB. 

The  initiative  could  approach  the  80-percent  goal  more  quickly  by 
eliminating  on-ground  accidents,  which  caused  more  fatal  accidents  in 
commercial  aviation  than  all  other  safety  problems  except  loss  of  control 
and  CFIT.  On-ground  accidents  caused  12  fatal  accidents  in  commercial 
aviation— 14  percent  of  the  total.  While  the  safety  problems  that  caused  on- 
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ground  accidents  merit  addressing,  the  safety  problems  that  Safer  Skies’ 
commercial  aviation  team  has  chosen  to  address  resulted  in  multiple  fatal 
accidents  and  many  more  fatalities.  For  this  reason,  the  initiative  will 
probably  have  more  impact  on  improving  the  safety  of  air  transportation 
for  the  majority  of  the  nation’s  passengers  than  addressing  other  safety 
problems,  such  as  on-ground  accidents,  whose  elimination  could  reduce 
the  fatal  accident  rate  more  but  would  save  fewer  lives. 

While  focusing  on  reducing  the  fatal  accident  rate  by  addressing  the  safety 
problems  that  caused  the  most  commercial  aviation  accidents,  the 
approach  taken  by  the  initiative  also  resulted  in  choices  that  recognized 
where  the  greatest  number  of  fatalities  have  occurred  or  could  occur.  The 
three  major  problems  addressed  by  the  initiative’s  commercial  aviation 
teams  (CFIT,  loss  of  control,  and  approach  and  landing)  accounted  for  57 
percent  of  the  1,756  fatalities  in  1988-97.  This  rises  to  66  percent  when  all 
six  safety  problems  on  the  commercial  aviation  agenda  are  considered.  The 
additional  small  commuter  accidents  that  are  to  be  addressed  by  general 
aviation  teams  account  for  another  5  percent  of  the  total  fatalities.  The  only 
other  safety  problems  that  resulted  in  hundreds  of  fatalities  were  sabotage 
and  fuel  tank  explosions.  The  initiative  did  not  focus  on  these  problems  for 
two  reasons.  First,  only  one  fatal  accident  resulted  from  each  of  these 
safety  problems  in  1988-97.  Second,  FAA  already  has  initiatives  under  way 
to  address  both  sabotage  and  fuel  tank  explosions.  The  initiative  did 
include  two  safety  problems  on  its  commercial  aviation  agenda  that  each 
accounted  for  only  about  1  percent  of  the  fatalities  in  U.  S.  commercial 
aviation  during  this  period.  However,  the  initiative  recognized  the  potential 
of  runway  incursions,  which  accounted  for  25  U.S.  fatalities,  to  result  in 
hundreds  of  fatalities.  While  weather  resulted  in  few  commercial  aviation 
accidents  and  16  fatalities,  the  commercial  aviation  steering  committee 
members  felt  that  the  problems  of  turbulence  and  icing  merited  attention. 
In  contrast,  the  12  on-ground  accidents  each  resulted  in  a  single  fatality 
that  together  accounted  for  fewer  than  1  percent  of  the  nation’s 
commercial  aviation  fatalities.  Eliminating  the  safety  problems  that  caused 
on-ground  fatalities  could  reduce  the  fatal  accident  rate  more  quickly  than 
eliminating  either  CFIT  or  approach  and  landing  accidents  that  involved 
large  commercial  aircraft.  The  commercial  aviation  steering  committee  has 
selected  safety  problems  that  will  help  reduce  fatalities,  as  well  as  the  fatal 
accident  rate.  (See  fig.  4.) 
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Figure  4:  Fatalities  in  U.S.  Commercial  Aviation  by  Safety  Problem,  1988-97 

600  Fatalities 


| _  _ _ 1  I _ I  I _ 

Commercial  aviation  teams  Referred  to  the  general  Other  FAA  efforts 

aviation  steering  committee 


l _ I 

Other 


Source:  FAA. 


The  Safer  Skies  Initiative 
Has  Adopted  the  80-Percent 
Goal  for  Commercial 
Aviation 


The  Safer  Skies  initiative  and  FAA  have  adopted  the  goal  of  reducing  the 
fatal  accident  rate  for  commercial  aviation  by  80  percent  by  2007. 
Specifically,  the  goal  is  to  reduce  the  fatal  accident  rate  for  commercial 
aviation  from  a  1994-96  baseline  of  0.037  fatal  accidents  per  100,000  flight 
hours  to  0.007  fatal  accidents  per  100,000  flight  hours  in  2007.  The  meaning 
of  this  goal  can  be  more  readily  understood  by  considering  the  current 
number  of  fatal  commercial  aviation  accidents  and  the  number  of  accidents 
projected  for  2007  if  further  safety  improvements  are  not  undertaken.  In 
1994-96,  the  United  States  averaged  six  fatal  commercial  aviation  accidents 
each  year.  Given  the  projected  growth  of  commercial  aviation,  we  estimate 
that  this  number  could  increase  to  nine  in  2007  if  safety  is  not  improved.  If 
the  initiative  achieves  the  goal  of  an  80-percent  reduction  in  the  fatal 
accident  rate  for  commercial  aviation,  we  estimate  that  the  number  of  fatal 
accidents  expected  in  2007  would  drop  to  two.  (See  table  6.) 
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Table  6:  Number  of  Past  and  Projected  Fatal  Accidents  by  Type  of  Aviation 
Operation 


Number  of  fatal  accidents 

Type  of  operation 

Annual 

average3 

Projected 
for  2007b 

Safer  Skies 
goal  for  2007c 

80-percent 

reduction*1 

Commercial 

aviation 

6 

9 

2 

2 

General  aviation 

380 

484 

350 

97 

Total 

386 

493 

352 

99 

aThe  annual  average  is  for  the  baseline  years  chosen  by  FAA  and  Safer  Skies  steering 
committees  in  establishing  their  goals:  1994-96  for  commercial  aviation  and  1996-98  for 
general  aviation. 

‘To  project  the  number  of  fatal  accidents  likely  in  2007,  we  used  FAA’s  most  recent  projected 
growth  rates  for  flight  hours  in  commercial  aviation  and  general  aviation,  which  were 
released  in  March  2000.  The  updated  growth  rate  for  general  aviation  was  larger  than  the 
earlier  growth  rate  available  to  the  general  aviation  steering  committee,  which  projected  437 
fatal  accidents  in  general  aviation  for  2007. 

cGiven  FAA’s  most  recent  projected  growth  rates,  the  Safer  Skies  goal  for  commercial 
aviation  of  an  80-percent  reduction  in  the  fatal  accident  rate  would  likely  result  in  two  fatal 
accidents  in  2007.  The  general  aviation  steering  committee  set  a  goal  of  reducing  the 
number  of  fatal  general  aviation  accidents  to  350  in  2007. 

dFor  commercial  aviation,  we  computed  the  number  of  fatal  accidents  that  would  result 
from  decreasing  the  fatal  accident  rate  in  2007  by  80  percent.  For  general  aviation,  we 
computed  the  number  of  fatal  accidents  that  would  result  from  decreasing  by  80  percent  the 
number  of  such  accidents  projected  for  2007. 

Source:  GAO’s  analysis  of  data  from  FAA  and  NTSB. 


Accident  Data  and 
Other  Resources  Were 
Used  to  Identify  Safety 
Problems  That  Caused 
Many  Fatal  Accidents 
in  General  Aviation 


The  general  aviation  steering  committee  used  available  accident  data, 
safety  reports,  and  professional  expertise  in  aviation  to  identify  safety 
problems  that  caused  many  of  the  fatal  accidents  in  general  aviation.  The 
six  safety  problems  chosen  were  controlled  flight  into  terrain,  loss  of 
control,  aeronautical  decision-making,  runway  incursions,  weather,  and 
survivability.  Steering  committee  members  told  us  that  they  selected  these 
safety  problems  after  reviewing  the  available  data  on  general  aviation 
accidents  and  past  industry  and  government-sponsored  safety  reports  on 
general  aviation.  They  said  that  the  NTSB  accident  reports  were 
challenging  to  analyze  because  many  lacked  the  detail  needed  to  determine 
the  root  causes  of  accidents.  They  noted,  for  example,  that  most  general 
aviation  aircraft  are  not  equipped  with  such  key  equipment  as  flight  data 
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recorders  that  would  help  identify  the  safety  problems  that  caused  the 
accidents.  To  meet  these  additional  challenges,  FAA  developed  a  training 
course  tailored  to  the  needs  of  those  responsible  for  analyzing  general 
aviation  accidents.  Both  FAA  and  industry  members  attended  this  course 
before  starting  the  analysis  phase.  The  general  aviation  analysis  reports  on 
CFIT  and  weather  also  included  recommendations  to  address  problems 
with  the  quality  of  the  data  on  general  aviation  accidents.  In  response  to 
these  recommendations,  the  general  aviation  steering  committee  chartered 
a  team  in  April  2000  to  develop  strategies  to  (1)  provide  increased  detail 
about  factors  that  have  contributed  to  or  caused  general  aviation  accidents 
and  incidents  and  (2)  improve  the  quality  and  timeliness  of  estimates  of 
general  aviation  activity.  Members  of  the  steering  committee  told  us  that,  in 
cases  where  the  safety  problems  that  caused  the  fatal  accidents  were 
unclear,  they  used  their  experience  as  either  pilots  or  experts  in  general 
aviation  to  determine  the  possible  causal  factors  involved  in  the  accidents. 

Members  of  the  steering  committee  also  examined  past  industry  and 
government  reports  on  the  causes  of  general  aviation  accidents.  One  key 
report  was  the  Nall  Report,  a  report  on  general  aviation  accident  trends  and 
factors  published  annually  by  the  Aircraft  Owners  and  Pilots  Association's 
Air  Safety  Foundation.  According  to  the  1998  Nall  Report,  the  major  causes 
of  fatal  general  aviation  accidents  were  weather,  loss  of  control  or  other 
errors  during  flights  in  which  the  pilot  was  maneuvering  the  plane,  and 
accidents  on  approach  to  the  airport.  Another  key  report  was  FAA’s  study 
of  the  causes  of  general  aviation  CFIT  accidents.3  FAA’s  study  concluded 
that  CFIT  accidents  accounted  for  17  percent  of  the  general  aviation 
fatalities  and  32  percent  of  general  aviation  accidents  in  weather  conditions 
requiring  pilots  to  have  instrument  ratings  to  fly. 

Steering  committee  members  also  told  us  that  several  reports  indicated 
growing  problems  with  runway  incursions  involving  general  aviation 
aircraft.  For  example,  a  study  by  DOT’S  Office  of  Inspector  General  showed 
that  general  aviation  pilots  caused  the  majority  of  runway  incursions 
attributable  to  pilot  error  in  1990-96.4  Members  of  the  steering  committee 
told  us  that  they  also  decided  to  address  survivability  in  an  effort  to 


3 General  Aviation  Accidents,  1983-1994:  Identification  of  Factors  Related  to  Controlled- 
Flight-Into  Terrain  (CFIT)  Accidents,  U.S.  DOT.  FAA  (DOT/FAA/ARR-lOO-97-2,  July  1997). 

4 Runway  Incursion  Program,  DOT,  FAA  (AV- 1998-075,  Feb.  9, 1998). 
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decrease  the  number  of  fatalities  among  those  who  survive  the  impact  of  a 
crash  but  ultimately  die  from  their  injuries. 

Although  the  data  available  on  general  aviation  accidents  are  less  detailed 
than  those  available  on  commercial  aviation  accidents,  the  general  aviation 
problems  the  initiative  plans  to  address  represent  reasonable  choices.  Most 
of  the  safety  problems  chosen  have  been  identified  in  past  safety  reports 
and  NTSB  accident  reports  as  major  causes  of  fatal  accidents  in  general 
aviation.  These  include  weather,  loss  of  control,  CFIT,  and  runway 
incursions.  Aeronautical  decision-making  has  also  been  cited  repeatedly  as 
a  factor  in  such  safety  problems  as  weather,  when  pilots  exercise  judgment 
about  whether  to  depart  or  turn  back  when  faced  with  potential  danger.  In 
addition,  aeronautical  decision-making  includes  those  decisions  made 
relating  to  aircraft  maintenance.  Most  of  the  Safer  Skies  participants,  FAA 
officials,  and  aviation  experts  we  interviewed  concurred  that  the  six 
general  aviation  safety  problems  to  be  addressed  by  the  initiative  are 
reasonable  ones  that  will  help  to  reduce  the  fatal  accident  rate. 


The  Safer  Skies  Initiative 
Adopted  a  Goal  of  Reducing 
Fatal  Accidents  in  General 
Aviation  to  350  in  2007 


Although  both  the  White  House  and  congressional  commissions  on  aviation 
safety  called  for  an  80-percent  reduction  in  the  nation’s  fatal  accident  rate, 
FAA  and  the  Safer  Skies  initiative  applied  this  goal  only  to  commercial 
aviation  and  adopted  a  less  aggressive  accident  reduction  goal  for  general 
aviation.  The  goal  is  to  reduce  the  number  of  fatal  general  aviation 
accidents  to  350  in  2007.  This  represents  a  20-percent  reduction  in  the 
number  of  fatal  accidents  that  would  likely  result  from  projected  growth  in 
general  aviation.  Because  general  aviation  accounted  for  98  percent  of  U.S. 
fatal  accidents  in  1988-97,  the  goal  of  an  80-percent  reduction  in  the 
nation’s  fatal  accident  rate  set  forth  by  the  two  major  aviation  commissions 
is  unreachable  if  these  fatal  accidents  are  not  greatly  reduced.  The 
congressionally  mandated  commission  on  aviation  safety  discussed  the 
fatal  accident  rates  for  all  kinds  of  aviation  operations,  including  general 
aviation.  Because  this  commission  did  not  explicitly  apply  the  80-percent 
goal  to  general  aviation,  it  remains  unclear  whether  it  intended  the  goal  to 
apply  to  general  aviation  as  well  as  commercial  aviation.  The  goal 
adopted — 350  fatal  accidents — contrasts  sharply  with  the  97  fatal  accidents 
that  would  likely  result  if  the  80-percent  goal  were  achieved.  (See  table  6.) 


The  steering  committee  did  not  adopt  the  80-percent  goal  for  general 
aviation  because  of  strong  objections  from  the  general  aviation  community. 
Representatives  of  the  general  aviation  community  argued  that,  given  the 
varied  experience  levels  of  its  pilots,  reducing  fatal  accidents  by  80  percent 
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would  be  impossible  without  grounding  the  fleet.  One  general  aviation 
representative  said  that  there  was  a  prevailing  concern  in  the  general 
aviation  community  that  any  agreement  on  a  solid  goal  would  lead  to  more 
regulation  and  less  growth.  In  addition,  these  representatives  objected  to 
establishing  a  goal  that  involved  a  fatal  accident  rate.  The  fatal  accident 
rate  for  general  aviation  is  calculated  by  dividing  the  number  of  fatal 
accidents  by  the  number  of  flight  hours.  Data  on  general  aviation  flight 
hours  are  estimated  using  an  annual  survey  of  general  aviation  operators 
conducted  by  FAA.  Response  to  the  survey  is  voluntary.  Because  the  flight 
hours  are  estimated  on  the  basis  of  this  survey,  representatives  of  the 
general  aviation  community  questioned  the  reliability  of  these  data  and 
expressed  concern  about  using  flight  hours  to  calculate  past  and  future 
fatal  accident  rates.  As  a  result,  the  Safer  Skies  steering  committee  for 
general  aviation  agreed  not  to  use  the  survey  data  on  flight  hours  to 
calculate  a  fatal  accident  rate  until  the  data  are  more  reliable.  Instead,  the 
accident  reduction  goal  for  general  aviation  was  expressed  in  terms  of  the 
number  of  fatal  accidents,  rather  than  the  fatal  accident  rate. 

To  set  its  goal  of  reducing  fatal  accidents  to  350,  the  general  aviation 
steering  committee  reviewed  available  data  on  fatal  accidents.  The  steering 
committee  found  the  number  had  declined  fairly  steadily  since  1990  in 
response  to  past  initiatives  to  improve  safety.  The  data  used  by  the  steering 
committee  showed  that,  in  1996-98,  an  average  of  379  fatal  general  aviation 
accidents  occurred  each  year.5  The  steering  committee  used  this  average 
and  the  1.6-percent  annual  growth  expected  in  general  aviation  to  project 
that  437  accidents  would  occur  in  2007  if  additional  safety  initiatives  were 
not  undertaken.  They  agreed  that  a  reduction  to  350  fatal  accidents  would 
be  achievable.  This  represents  a  20-percent  reduction  in  the  number  of  fatal 
accidents  that  they  estimated  would  occur  without  additional  safety 
initiatives  (437) .  According  to  projections  by  the  steering  committee  and 
the  general  aviation  community,  a  reduction  of  this  magnitude  would 
prevent  363  accidents  from  2000  through  2007. 

The  goal  of  reducing  the  number  of  fatal  accidents  to  350  in  2007  is 
probably  achievable,  but  this  goal  is  not  likely  to  push  the  general  aviation 
community  toward  more  safety  improvement  as  aggressively  as  it  could. 


5The  general  aviation  steering  committee  used  preliminary  NTSB  data  to  compute  the 
average  number  of  fatal  accidents  for  1996-98,  which  resulted  in  an  annual  average  of  379.  In 
table  6  and  throughout  our  report,  we  used  the  official  NTSB  accident  statistics  for  1999, 
which  were  released  after  the  steering  committee’s  projections.  This  resulted  in  a  slightly 
higher  annual  average  of  380  fatal  accidents  in  general  aviation. 
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We  believe  that  this  goal  is  achievable  for  two  reasons.  First,  although  the 
level  of  general  aviation  activity  has  increased,  the  number  of  fatal 
accidents  decreased  to  354  in  1999,  a  decrease  of  17  percent  since  1994. 
Both  FAA  and  industry  officials  attributed  this  decrease  in  part  to  ongoing 
safety  initiatives.  The  goal  of  379  accidents  established  for  each  of  the  next 
3  years  represents  a  7-percent  growth  in  the  current  number  of  fatal 
accidents.  Second,  the  goal  of  350  accidents  set  for  2007  is  only  4  fewer 
fatal  accidents  than  occurred  last  year.  Hence,  the  long-term  goal  is 
achievable  if  the  general  aviation  community  is  able  to  hold  its  number  of 
fatal  accidents  steady  as  its  air  traffic  grows  by  an  expected  2.2  percent  per 
year  in  the  coming  decade. 

We  recognize  that  an  80-percent  reduction  in  fatal  accidents  is  probably  not 
achievable  in  general  aviation  at  this  time  because  of  the  diversity  in  pilots’ 
experience  levels,  aircraft  types,  and  operating  environments.  However,  we 
believe  that  a  more  aggressive  goal  would  encourage  greater  efforts  by 
general  aviation  operators,  manufacturers,  associations,  and  FAA  to  make 
safety  improvements  in  general  aviation  operations  that  could  save  lives. 


Improving  Cabin  Safety 
Is  Important  but  Will 
Have 

Little  Impact  on 
Lowering  the  Fatal 
Accident  Rate 


Improving  cabin  safety  is  unlikely  to  have  much  impact  on  reducing  the 
overall  fatal  accident  rate.  In  contrast  to  the  safety  problems  addressed  by 
the  commercial  and  general  aviation  steering  committees,  the  safety 
problems  addressed  by  the  cabin  safety  steering  committee  have  not 
resulted  in  numerous  fatalities,  and  few  data  are  available  on  any  injuries 
that  result  from  these  problems.  The  Safer  Skies  initiative  identified  only 
two  fatalities  in  U.S.  commercial  aviation  in  1988-97  related  to  cabin  safety 
problems.6  While  passengers  and  crew  have  been  injured  in  the  cabin 
environment,  few  data  exist  on  these  incidents  because  air  carriers  are  not 
required  to  report  such  incidents  unless  they  involve  a  serious  injury  or 
fatality.  The  study  of  cabin  safety  problems  thus  relies  more  on  information 
shared  by  flight  attendants  and  air  carriers  than  on  analysis  of  the  limited 
data  available.  Because  cabin  safety  resulted  in  few  fatalities  and  affords 
few  data  for  analysis,  it  is  arguable  whether  it  was  appropriate  to  include 
cabin  safety  issues  in  an  initiative  directed  at  reducing  the  fatal  accident 
rate  through  a  data-driven  analysis  of  safety  problems. 


6Both  accidents  resulted  in  the  death  of  a  passenger  who  was  not  secured  by  a  seat  belt 
when  the  aircraft  encountered  turbulence.  The  commercial  aviation  steering  committee  lists 
weather  as  the  safety  problem  that  caused  these  two  accidents. 
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Although  not  appropriate  for  Safer  Skies’  focus  on  the  safety  problems  that 
caused  fatal  aviation  accidents,  cabin  safety  issues  are  an  appropriate  topic 
for  FAA  to  address  jointly  with  the  aviation  industry.  NTSB  and  flight  crews 
have  raised  concerns  about  the  potential  for  injuries  and  fatalities  in  the 
cabin.  FAA  and  industry  were  jointly  studying  cabin  safety  problems  before 
the  initiative  was  announced.  The  safety  problems  under  study  included 
those  involving  child  restraint  systems,  passenger  seatbelt  use,  passenger 
interference  with  crew,  and  carry-on  baggage.  Concerns  about  these  safety 
problems  are  not  new.  For  example,  NTSB  has  long  advocated  FAA’s 
requiring  the  use  of  child  restraints  for  passengers  under  the  age  of  2.  NTSB 
was  concerned  enough  about  the  use  of  child  restraints  to  launch  a 
campaign  aimed  at  making  parents  aware  of  the  benefit  of  putting  children 
in  approved  child  restraint  systems  and  to  declare  1999  as  the  “year  of  child 
transportation  safety.”  Similarly,  representatives  of  air  carrier  crews  have 
expressed  concern  that  the  incidents  of  passengers  interfering  with  crew 
members  are  increasing. 


Additional  Work  on 
Future  Hazards  Could 
Help  Anticipate  and 
Prevent  Fatal 
Accidents 


In  December  1997,  the  congressionally  mandated  commission  on  aviation 
safety  recommended  that  FAA  and  the  aviation  industry  jointly  develop  a 
strategic  plan  to  improve  aviation  safety  and  that  the  process  “begin  with 
analysis  of  both  previous  and  potential  failures  to  meet  safety 
expectations.”  These  failures  include  accidents,  incidents,  insight  from 
flight  operational  data,  and  aviation  system  changes.  The  analysis  of  the 
causes  of  past  accidents  provides  insights  into  safety  problems  that  exist 
within  the  current  aviation  system,  while  the  analysis  of  aviation  system 
changes  can  help  anticipate  future  hazards  that  may  arise  from  such 
changes  as  growth  and  technological  advances  (e.g.,  vertical  takeoff  and 
landing  by  aircraft).  The  approaches  to  the  analysis  of  past  safety  problems 
and  future  hazards  are  distinct.  A  data-driven  approach  is  particularly 
useful  for  analyzing  the  safety  problems  that  caused  past  fatal  accidents. 
Data  on  nonfatal  accidents  and  incidents  can  also  be  used  to  identify  and 
address  safety  problems  that  did  not  result  in  fatalities  but  could  have.  The 
data-driven  approach  is  based  on  the  assumption  that  identifying  a  problem 
is  possible  where  historical  data  are  available.  While  this  approach  can  be 
used  to  address  the  safety  problems  in  the  current  operating  environment, 
other  types  of  analyses  may  be  more  useful  for  anticipating  and  preventing 
the  safety  problems  that  could  result  in  new  types  of  fatal  accidents.  For' 
example,  the  anticipated  growth  in  air  traffic  will  lead  to  more  congestion 
around  airports,  increasing  the  possibility  of  runway  incursions  and  midair 
collisions  near  airports.  Anticipating  how  changes  in  the  aviation  industry 
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may  increase  existing  safety  problems  or  bring  about  new  ones  can  better 
position  both  FAA  and  the  aviation  industry  to  prevent  accidents. 

While  FAA,  Safer  Skies,  and  industry  groups  have  made  progress  in  the 
analysis  of  the  causes  of  past  accidents  and  incidents,  efforts  to  analyze 
and  anticipate  future  hazards  are  more  preliminary.  The  Joint  Safety 
Strategy  Initiative  in  Europe7  has  formed  a  work  group  to  develop  a  method 
for  examining  future  hazards.  A  number  of  FAA  staff  participate  in  this 
work  group,  which  should  facilitate  the  cooperative  exchange  of  ideas  and 
information  on  this  topic.  As  of  April  2000,  the  Safer  Skies  initiative  had  not 
established  a  process  for  analyzing  future  hazards.  A  systematic  analysis  of 
the  changes  occurring  in  the  aviation  industry  could  enhance  Safer  Skies’ 
ongoing  efforts  to  reduce  the  fatal  accident  rate.  Several  of  the  aviation 
experts  interviewed  suggested  that  the  initiative  could  benefit  from  going 
beyond  the  analysis  of  data  on  past  accidents  to  consider  safety  problems 
that  may  arise  from  rapid  changes  in  the  aviation  operating  environment. 
Participants  on  Safer  Skies’  commercial  aviation  steering  committee  also 
indicated  that  while  data-driven  approaches  are  helpful,  it  is  also  important 
to  consider  future  hazards.  FAA’s  Director  of  the  Aircraft  Certification 
Service8  said  that  the  initiative’s  first  priority  was  to  understand  and 
eliminate  the  safety  problems  that  caused  past  accidents  but  that  the 
commercial  aviation  steering  committee  also  plans  to  address  future 
hazards  and  recently  added  this  topic  to  its  agenda  for  consideration. 
Because  work  on  future  hazards  could  help  anticipate  and  prevent  fatal 
accidents,  this  topic  is  important  for  the  Safer  Skies  steering  committees  to 
address,  especially  as  it  applies  to  commercial  aviation.  Coordinating  this 
effort  with  the  work  initiated  by  European  and  FAA  staff  on  future  hazards 
should  help  avoid  duplication  of  effort  and  foster  awareness  of  and 
solutions  to  these  potential  problems  internationally. 


Conclusions  The  premise  of  both  the  White  House  and  congressional  commissions  on 

aviation  safety  was  that  data  on  past  and  possible  future  causes  of 
accidents  could  be  used  to  focus  resources  on  substantially  reducing  the 
fatal  accident  rate.  While  the  Safer  Skies  initiative  has  made  significant 


7The  Joint  Safety  Strategy  Initiative  includes  members  from  European  aviation 
manufacturers,  associations,  and  regulators. 

8The  Director  of  the  Aircraft  Certification  Service  co-chairs  the  commercial  aviation 
steering  committee. 
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strides,  it  has  not  yet  carried  out  this  mandate  as  fully  as  it  could.  The  six 
safety  problems  that  the  initiative  is  addressing  accounted  for  almost  80 
percent  of  the  fatal  accidents  in  commercial  aviation  in  1988-97.  Our  review 
showed  that  the  initiative  and  FAA  have  work  under  way  to  address  these 
and  other  safety  problems  in  commercial  aviation.  However,  the  initiative 
has  not  challenged  all  sectors  of  the  aviation  community  to  push 
aggressively  for  safety  improvements.  Although  the  initiative  has  adopted 
the  challenging  goal  of  reducing  the  fatal  accident  rate  for  commercial 
aviation  by  80  percent  by  2007,  general  aviation  is  not  being  asked  to  set  a 
similarly  challenging  goal.  While  an  80-percent  reduction  in  fatal  accidents 
is  probably  not  achievable  in  general  aviation  at  this  time,  the  goal  adopted 
by  the  initiative  does  not  push  the  general  aviation  community  toward 
implementing  the  kinds  of  interventions  that  could  substantially  lower  the 
fatal  accident  rate.  A  more  rigorous  goal  would  encourage  greater  efforts 
by  general  aviation  operators,  manufacturers,  associations,  and  FAA  to 
make  needed  safety  improvements.  In  addition,  many  of  the  interventions 
developed  to  improve  general  aviation  safety  could  also  benefit  small 
commuter  operators  and  pilots,  but  this  benefit  will  not  be  realized  without 
a  systematic  way  of  ensuring  that  training  and  other  interventions  are  also 
directed  at  small  commercial  aviation  operations.  Finally,  the  Safer  Skies 
initiative  and  most  aviation  safety  studies  to  date  have  focused  on  the 
causes  of  past  accidents.  While  analyses  of  accident  data  are  useful  for 
determining  the  causes  of  past  accidents,  reducing  fatal  accidents  during  a 
period  of  rapid  growth  in  aviation  will  probably  require  the  analysis  of  the 
changing  aviation  environment  to  anticipate  future  safety  problems. 
Preliminary  international  efforts  have  been  initiated  to  consider  future 
hazards,  and  integrating  these  efforts  with  Safer  Skies’  work  would 
enhance  the  initiative’s  efforts  to  reduce  the  fatal  accident  rate. 


Recommendations 


To  further  reduce  the  nation’s  fatal  accident  rate  and  save  lives  in  the  type 
of  aviation  operation  that  causes  the  most  fatal  accidents  and  fatalities,  we 
recommend  that  the  Secretary  of  Transportation  direct  the  FAA 
Administrator  to  work  with  the  general  aviation  community  to 

•  set  a  more  challenging  goal  for  reducing  the  number  of  fatal  general 
aviation  accidents  by  2007, 

•  set  interim  goals  to  assess  progress  toward  this  new  goal,  and 

•  ensure  that  training  and  other  interventions  that  emerge  from  general 
aviation  teams  are  communicated  to  small  commuter  operators  and 
pilots  who  may  benefit  from  them. 
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DOT  and  FAA  officials  concurred  with  our  recommendations  aimed  at 
setting  a  more  challenging  interim  goal  and  long-term  goals  for  general 
aviation  and  said  that  they  planned  to  do  so  in  the  future.  However,  the 
officials  noted  that  existing  general  aviation  accident  data  are  too 
inaccurate  to  be  used  as  the  basis  for  setting  an  accident  reduction  goal. 
The  general  aviation  steering  committee  has  established  a  work  group  to 
recommend  ways  to  improve  the  quality  of  general  aviation  data.  The 
officials  stated  that  FAA  and  the  general  aviation  community  would  review 
the  accident  reduction  goal  when  the  quality  of  the  data  improves. 

DOT  and  FAA  officials  disagreed  with  our  recommendation  aimed  at 
ensuring  that  training  and  other  interventions  emerging  from  general 
aviation  teams  are  communicated  to  small  commuter  operators  because 
they  believe  that  mechanisms  already  exist  to  do  this.  The  officials 
explained  that  a  number  of  associations  representing  smaller  commuter 
aircraft  participate  on  the  general  aviation  steering  committee  and  on  its 
analysis  and  implementation  teams.  These  organizations  provide  conduits 
for  transmitting  interventions  developed  by  the  general  aviation  teams  to 
small  commuter  operators.  We  agree  that  these  organizations  may  facilitate 
the  transfer  of  safety  interventions  developed  by  the  general  aviation  teams 
to  small  commuter  air  carriers.  However,  it  will  be  difficult  to  achieve  the 
mandated  80-percent  reduction  in  commercial  aviation  fatalities  without 
systematic  improvements  in  the  safety  record  of  small  commuter  air 
carriers,  which  accounted  for  28  percent  of  fatal  commercial  aviation 
accidents.  We  believe  that  Safer  Skies  would  benefit  from  a  systematic  plan 
for  ensuring  that  interventions  developed  by  general  aviation  teams  are 
communicated  to  and  implemented  by  small  commuter  operators.  For  this 
reason,  we  did  not  modify  or  delete  our  recommendation. 

DOT  and  FAA  officials  disagreed  with  our  recommendation  calling  for  an 
analysis  of  future  safety  problems  arising  from  the  rapid  growth  and 
changes  in  aviation.  The  officials  noted  that  efforts  involving  FAA,  Safer 
Skies,  and  the  European  aviation  industry  are  already  under  way  to  address 
future  hazards  in  aviation.  On  the  basis  of  the  information  presented  by 
DOT  and  FAA  officials,  we  withdrew  this  recommendation. 
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Joint  FAA  and  industry  teams  have  started  work  on  13  of  the  16  problems 
being  addressed  by  the  initiative.  A  two-part  process  has  been  developed 
for  use  by  these  teams  to  first  analyze  accident  and  incident  data  and  then 
to  use  that  analysis  to  identify,  select,  and  implement  safety  interventions 
to  help  prevent  accidents  in  the  future.  That  process  is  reasonable  and  has 
allowed  FAA  and  industry  groups  to  reach  consensus  on  how  to  address 
safety  problems  identified  under  the  initiative.  This  process  was  not  used 
to  address  cabin  safety  problems  because  the  cabin  safety  steering 
committee  had  already  begun  its  work  before  the  process  was  developed. 
The  Safer  Skies  teams  have  made  progress  primarily  in  those  areas  that  had 
been  studied  extensively  in  the  past  for  which  widely  supported 
recommendations  already  existed.  The  interventions  recommended  for  five 
problems  are  now  being  implemented:  uncontained  engine  failure  and 
CFIT  in  commercial  aviation  and  passenger  seatbelt  use,  child  restraint 
systems,  and  carry-on  baggage  in  cabin  safety.  The  process  being  used  will 
require  more  extensive  analysis  in  the  future  as  teams  begin  to  address 
safety  problems  that  have  not  been  studied  previously.  Finally,  the  success 
of  the  interventions  that  the  Safer  Skies  teams  have  chosen  to  address 
these  long-standing  safety  problems  depends  in  part  on  effective 
implementation.  Our  past  work  has  shown  that  FAA  does  not  consistently 
follow  through  on  implementing  key  safety  recommendations. 
Furthermore,  FAA  and  the  aviation  industry  began  implementing  some  of 
the  Safer  Skies  safety  interventions  before  having  a  process  in  place  to 
track  their  progress.  The  initiative  has  developed  a  process  for  tracking  the 
implementation  of  interventions  to  improve  safety  in  commercial  aviation. 
However,  the  implementation  of  Safer  Skies’  interventions  is  not  assured 
because  the  tracking  system  for  commercial  aviation  is  not  sufficiently 
detailed  to  assess  progress  in  implementing  interventions.  Furthermore, 
the  cabin  safety  steering  committee  implemented  its  interventions  without 
having  a  tracking  process  in  place,  and  the  general  aviation  steering 
committee  is  working  toward  the  final  approval  of  interventions  to  address 
two  safety  problems  without  having  a  tracking  process.  Without  a  complete 
tracking  process,  FAA  and  the  industry  cannot  ensure  that  the  initiative  will 
.improve  aviation  safety  in  each  of  these  areas. 
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The  Safer  Skies 
Methodology  Is  Based 
on  Previous  Efforts  to 
Identify  Safety 
Problems 


For  the  Safer  Skies  initiative,  FAA  and  the  aviation  industry  jointly 
developed  a  two-part  process  to  analyze  accident  data  and  then  to  choose 
from  among  the  possible  interventions.  This  process  grew  out  of  a  previous 
FAA  effort  that  used  a  data-driven  approach  to  identify  threats  to  aviation 
safety  and  develop  interventions  to  address  those  threats.  During  the  first 
part  of  this  process,  an  analysis  team  reviews  accident  data  to  determine 
what  went  wrong,  why  it  went  wrong,  and  what  interventions  might  be  the 
most  effective  in  preventing  similar  accidents  in  the  future.  The  second 
part  of  the  process  involves  another  team  that  assesses  the  feasibility  of 
each  potential  intervention,  prioritizes  the  interventions  on  the  basis  of 
their  effectiveness  and  feasibility,  and  submits  plans  for  implementing 
projects  to  the  steering  committee  for  approval.  However,  as  we  discuss 
later  in  this  chapter,  the  steering  committee  addressing  cabin  safety 
problems  did  not  use  this  process. 


The  Initiative  Uses  a  Two- 
Part  Process  to  Analyze 
Data  and  Identify 
Interventions 


The  initiative  uses  a  two-part  process  to  analyze  data  and  identify 
interventions  to  address  safety  problems  in  commercial  aviation  and 
general  aviation.  This  process  is  modeled  on  an  analysis  of  the  most 
significant  threats  to  aviation  safety  conducted  in  1997  by  staff  from  FAA’s 
Aircraft  Certification  Service.  The  two-part  process  was  developed  for  use 
by  the  teams  addressing  safety  problems  in  commercial  aviation  but  has 
also  been  used  by  the  general  aviation  teams  with  some  modifications. 
Under  the  process,  the  steering  committee  forms  an  analysis  team  for  each 
aviation  safety  problem.  The  team,  which  includes  members  from  FAA  and 
the  aviation  industry,  reviews  accident  data,  determines  accident  causes, 
and  identifies  possible  interventions  to  prevent  future  accidents.  For 
selected  accidents,  the  team  develops  a  detailed  sequence  of  events  that 
includes  the  actions  by  pilots  and  air  traffic  controllers  as  well  as  any 
system  or  equipment  failure.  The  team  determines  what  went  wrong  and 
why  and  then  considers  various  interventions  that  could  have  prevented 
the  accident.  In  its  final  report,  the  analysis  team  ranks  all  of  the  identified 
interventions  by  their  effectiveness  in  preventing  similar  accidents  and 
presents  them  to  the  steering  committee  for  further  action. 

Once  the  analysis  team  completes  its  work  on  a  safety  problem,  the 
steering  committee  forms  a  second  team  to  assess  the  feasibility  of 
implementing  the  interventions  suggested  by  the  analysis  team.  The 
implementation  team  assesses  feasibility  in  six  areas:  the  cost  of  the 
intervention;  the  time  needed  to  implement  it;  whether  it  requires 
regulatory  changes;  technical  feasibility;  the  practicality  of  the  project 
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within  the  operating  environment  or  the  nationwide  aviation  system;  and 
political  feasibility.  The  implementation  team  prioritizes  the  interventions 
by  both  effectiveness  and  feasibility  and  then  presents  the  resulting 
prioritized  list  to  the  steering  committee.  Once  the  steering  committee 
initially  approves  an  intervention,  the  implementation  team  develops  a 
detailed  project  plan  for  implementation  that  is  sent  to  the  steering 
committee  for  final  approval.  Once  detailed  plans  are  approved,  the 
interventions  are  then  implemented  by  the  responsible  organizations. 

The  general  aviation  teams  have  made  some  modifications  to  the  analysis 
process  initially  developed  for  use  by  the  commercial  aviation  teams. 
Although  the  first  commercial  aviation  analysis  team  considered  feasibility 
as  well  as  effectiveness,  the  two-part  process  ultimately  approved  for 
commercial  aviation  teams  considers  only  effectiveness  at  the  analysis 
stage.  Any  consideration  of  such  matters  as  cost  and  the  need  for 
developing  new  regulations  is  left  to  the  implementation  team.  In  contrast, 
the  general  aviation  analysis  teams  consider  both  effectiveness  and 
feasibility.  Our  review  of  the  general  aviation  analysis  team’s  reports  for 
CFIT  and  weather  confirmed  that  such  feasibility  criteria  as  cost  and  the 
need  for  new  regulations  have  been  considered  far  earlier  in  the 
assessment  of  general  aviation  interventions  than  in  the  process  now  used 
by  commercial  aviation  teams.  While  other  feasibility  factors  are  also 
considered,  cost,  the  avoidance  of  interventions  that  would  require  new 
regulations,  and  acceptability  to  the  general  aviation  community  have 
weighed  heavily  in  the  choice  of  interventions  to  address  general  aviation 
safety  problems.  In  emphasizing  cost  and  acceptability  to  the  aviation 
community,  the  general  aviation  teams  have  selected  training  and  other 
interventions  that  will  be  more  affordable  to  general  aviation  pilots. 


The  Cabin  Safety  Team  Used 
a  Different  Approach 


While  the  initiative  is  using  a  systematic,  defined  approach  to  consider 
ways  to  address  safety  problems  in  commercial  and  general  aviation,  a 
different  approach  was  used  to  address  cabin  safety  problems.  Several 
months  before  the  announcement  of  the  Safer  Skies  initiative,  FAA 
established  the  Partners  in  Cabin  Safety  (PICS)  team  to  provide 
information  to  the  public  about  four  cabin  safety  problems:  passenger 
interference  with  flight  crews,  the  safety  benefits  of  greater  use  of  seat 
belts  by  passengers,  the  safety  benefits  of  child  safety  restraints,  and 
potential  safety  issues  arising  from  the  stowage  of  carry-on  baggage. 
According  to  PICS  team  members,  FAA  identified  these  problems  before 
assigning  them  as  tasks  to  the  team  in  January  1998.  Team  members 
discussed  such  additional  issues  as  in-flight  medical  emergencies  and  cabin 
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air  quality  but  settled  on  the  four  that  were  eventually  included.  Unlike  the 
commercial  and  general  aviation  teams,  the  PICS  team  limited  the  possible 
interventions  to  ones  that  did  not  require  that  FAA  create  new  regulations, 
a  process  that  was  viewed  by  some  participants  as  too  slow  and  unlikely  to 
result  in  consensus  among  various  industry  and  government  participants. 
Consequently,  the  PICS  team  focused  on  interventions  that  involved 
educating  passengers. 

The  four  cabin  safety  problems  addressed  differ  in  several  important  ways 
from  those  safety  problems  addressed  by  the  Safer  Skies  teams  in 
commercial  and  general  aviation.  First,  the  cabin  safety  problems  resulted 
in  only  two  fatalities  in  U.S.  commercial  aviation  from  1988  through  1997, 
both  involving  passengers  not  using  their  seat  belts  when  the  aircraft 
encountered  turbulence.  In  contrast,  during  the  same  period  of  time,  there 
were  more  than  9,800  fatalities  in  all  commercial  and  general  aviation 
accidents.  Second,  air  carriers  are  not  required  to  maintain  or  submit  data 
on  cabin  safety  incidents  unless  they  involve  fatalities  or  serious  injuries. 
Since  only  limited  historical  data  on  cabin  safety  accidents  and  injuries 
were  available  for  analysis,  the  PICS  team  did  not  conduct  a  causal  analysis 
as  has  been  done  by  the  analysis  and  implementation  teams  for  both 
commercial  and  general  aviation. 

The  PICS  team  disbanded  in  January  1999  after  it  completed  the 
development  of  passenger  education  materials.  As  part  of  its  passenger 
education  efforts,  the  team  distributed  brochures  on  child  restraint 
systems  from  a  previous  campaign  by  FAA.  In  addition,  the  Luggage  and 
Leather  Goods  Manufacturers  of  America,  along  with  FAA,  developed  a 
brochure  addressing  carry-on  baggage  concerns,  which  the  team  members 
were  asked  to  distribute  to  airlines,  luggage  stores,  and  airports.  It  was  also 
put  on  FAA’s  World  Wide  Web  site  for  further  distribution  by  interested 
parties.  Steering  committee  members  and  FAA  officials  also  told  us  that  the 
team  worked  with  air  carriers  to  develop  additional  cabin  announcements 
for  the  stowage  of  carry-on  baggage  and  the  importance  of  seat  belt  usage. 
Finally,  the  PICS  team  developed  a  passenger  safety  checklist  for 
publication  on  FAA’s  Web  site,  which  addressed  passenger  interference 
with  flight  crews,  seat  belt  usage,  child  restraint  systems,  and  carry-on 
baggage.  This  checklist,  however,  is  not  currently  available  on  FAA’s  Web 
site.  According  to  an  official  at  FAA’s  Flight  Standards  Service,  the 
passenger  safety  checklist  project  is  on  hold  until  the  agency  appoints  a 
new  national  resource  specialist  for  cabin  safety  who  will  review  the 
document  before  it  is  made  available  to  the  public. 
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The  Safer  Skies 
Initiative  Has  Made  the 
Most 

Progress  With 
Problems  Studied 
Previously 


Since  the  FAA  Administrator  announced  the  Safer  Skies  initiative  in  April 
1998,  work  has  started  on  9  of  the  1 2  safety  problems  to  be  addressed  in  the 
commercial  and  general  aviation  safety  areas.  Teams  have  made  the  most 
progress  in  selecting  interventions  for  safety  problems  when  they  could 
build  on  previous  studies  for  which  widely  supported  recommendations 
exist.  The  commercial  aviation  steering  committee  plans  to  have  work 
started  on  all  of  the  identified  problems  before  the  end  of  fiscal  year  2000, 
but  the  general  aviation  steering  committee  has  not  yet  determined  when 
work  on  three  of  its  six  problems  will  begin.  Table  7  shows  the  status  of  the 
work  on  each  of  the  12  safety  problems  to  be  addressed  in  commercial  and 
general  aviation  as  of  April  1,  2000. 
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Table  7:  Status  of  Safer  Skies’  Activities  for  Commercial  and  General  Aviation 


Analysis  team 

Implementation  team 

Team 

Final  report 

Team 

Final  report 

formed 

issued 

formed 

issued 

Commercial  aviation 

CFIT 

X 

X 

Xa 

X 

Loss  of  control 

X 

Uncontained  engine  failure15 

X 

X 

X 

X 

Runway  incursions0 

X 

Approach  and  landing 

X 

X 

Xa 

X 

Weather" 

Turbulence 

X 

Icing 

General  aviation 

CFIT 

X 

X 

X 

e 

Loss  of  control 

Runway  incursions 

X 

Aeronautical  decision-making 

Survivability 

Weather 

X 

X 

X 

e 

The  commercial  aviation  steering  committee  combined  the  implementation  teams  for  CFIT 
and  approach  and  landing  accidents  because  many  of  the  interventions  chosen  by  the 
analysis  teams  for  these  two  safety  problems  overlapped. 


The  joint  FAA/industry  team  working  on  uncontained  engine  failure  developed  the 
prototype  process  used  by  the  Safer  Skies  analysis  and  implementation  teams. 

The  analysis  activities  for  runway  incursions  include  both  commercial  and  general  aviation 
accidents  reviewed  by  a  joint  team. 

The  commercial  aviation  steering  committee  will  form  separate  teams  to  address  two 
weather  issues— turbulence  and  icing. 

The  general  aviation  CFIT  and  weather  reports  were  presented  to  us  as  final  reports. 
However,  in  responding  to  our  draft  report,  FAA  told  us  that  these  reports  had  not  received 
final  approval. 

Source:  GAO  ‘s  analysis  of  Safer  Skies’  data. 
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The  steering  committees  charged  with  developing  interventions  for  each  of 
the  safety  problems  first  formed  analysis  teams  to  work  on  problems  for 
which  major  studies  had  already  been  done  or  were  under  way.  The 
ongoing  and  completed  studies  conducted  by  FAA  and  the  industry 
provided  information  the  analysis  teams  could  use  to  identify  the  causes  of 
accidents  and  potential  interventions.  For  example,  the  Flight  Safety 
Foundation  had  completed  an  extensive  study  on  CFIT,  examining  over  250 
accidents  and  incidents  worldwide.  The  foundation  had  also  developed 
training  materials  for  pilots  and  made  other  recommendations  to  prevent 
CFIT  accidents.  In  another  instance,  the  team  analyzing  weather-related 
accidents  involving  general  aviation  aircraft  identified  1 1  safety  studies  that 
had  preceded  its  efforts,  all  of  which  recommended  interventions  similar  to 
the  ones  the  team  ultimately  identified.  FAA  participants  on  the  Safer  Skies 
commercial  aviation  steering  committee  told  us  that  beginning  with 
previously  studied  safety  problems  helped  team  members  make  progress  in 
developing  the  team’s  two-part  process  for  analyzing  data  and  identifying 
interventions  and  become  comfortable  with  the  analysis  and  selection 
process  before  moving  onto  more  complex  issues  that  may  involve  original 
research  and  analysis.  However,  this  approach  meant  that  work  on  another 
area  that  is  important  for  reducing  the  fatal  accident  rate  in  commercial 
aviation  did  not  start  work  until  September  1999 — 17  months  after  the 
initiative  s  announcement.  FAA  identified  loss  of  control  as  the  single 
largest  cause  of  fatal  commercial  aviation  accidents  involving  U.S. 
operators. 

To  date,  the  implementation  of  interventions  has  concentrated  mostly  in 
areas  for  which  analysis  and  implementation  were  well  under  way  or 
complete  when  the  initiative  began.  The  first  of  the  interventions  to  be 
implemented  addressed  uncontained  engine  failure:  FAA  issued  a  series  of 
airworthiness  directives  requiring  enhanced  inspections  of  high-speed 
rotating  parts  in  certain  jet  engines.1  The  directives  require  industry 
maintenance  personnel  to  perform  additional,  more  detailed  inspections  to 
check  for  cracks  and  other  signs  of  irregularities  whenever  an  engine  is 
disassembled  for  overhaul  or  maintenance.  According  to  staff  at  FAA’s 
Engine  and  Propeller  Directorate,  these  directives  affect  more  than  90 
percent  of  the  jet  engines  that  U.S.  airlines  currently  use.  FAA  and  the 


‘FAA  issues  airworthiness  directives  to  address  unsafe  mechanical  conditions  that  surface 
after  an  aircraft  has  been  certified  and  in  use.  The  directives  contain  FAA’s  requirements  for 
airlines  to  correct  unsafe  aircraft  conditions  that  have  occurred  or  are  likely  to  occur  in 
aircraft  of  the  same  design. 
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industry  are  also  taking  steps  to  implement  an  intervention  endorsed  by  the 
commercial  aviation  team  examining  CFIT.  The  team  has  recommended 
that  enhanced  navigational  equipment  be  installed  on  new  and  existing 
aircraft  to  warn  pilots  of  impending  crashes.  Air  carriers  began  installing 
the  enhanced  navigational  equipment  to  prevent  CFIT  accidents  in  their 
aircraft  before  FAA  issued  the  final  rule  requiring  that  the  equipment  be 
installed  in  the  commercial  fleet.  Specifically,  air  carriers  began  installing 
the  enhanced  navigational  equipment  to  prevent  CFIT  accidents  in  their 
aircraft  before  FAA  issued  its  final  rule  in  March  2000  and  before  the 
commercial  aviation  team  working  on  CFIT  issued  its  final  report  in  June 
2000.  This  equipment  is  now  being  included  on  some  new  aircraft,  and 
airlines  had  equipped  about  4,000  aircraft  already  in  service  with  the  new 
technology  by  December  31, 1999. 

The  timetable  for  analysis  and  implementation  teams  addressing  the 
problems  included  under  the  initiative  has  changed  since  the  initiative  was 
announced  in  April  1998.  According  to  the  chairs  of  the  Safer  Skies  steering 
committees,  some  of  these  schedule  changes  occurred  because  the 
analysis  process  took  longer  than  anticipated.  In  other  cases,  changes  to 
the  analysis  approach  required  rescheduling  Safer  Skies’  efforts.  For 
example,  FAA  officials  explained  that  the  final  report  date  for  the 
commercial  aviation  CFIT  implementation  team  was  rescheduled  after  the 
steering  committee  decided  that  combining  the  CFIT  and  approach  and 
landing  teams  for  the  implementation  analysis  made  sense  because  of 
overlap  in  the  interventions  they  had  identified.  Several  high-priority 
interventions  to  address  CFIT  accidents  in  commercial  aviation  were, 
however,  forwarded  to  the  steering  committee  for  final  approval  and 
implementation  without  waiting  for  the  implementation  team’s  final  report. 
An  FAA  co-chair  of  the  general  aviation  steering  committee  told  us  that 
they  changed  the  start  dates  for  several  of  the  general  aviation  teams 
because  general  aviation  accidents  are  more  numerous  than  commercial 
aviation  accidents  and  analyzing  them  proved  more  time-consuming  than 
anticipated.  This  FAA  official  also  said  that  some  general  aviation  groups 
participating  in  the  initiative  do  not  have  enough  people  or  resources  to 
serve  on  multiple  teams  simultaneously.  While  we  believe  that  these 
decisions  were  justified,  they  also  effectively  mean  that  the  interventions  to 
resolve  some  key  safety  problems  will  not  be  identified  or  implemented 
until  later  than  originally  anticipated. 
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Early  Experience 
Indicates  That  Future 
Problems  Will  Require 
More  Analysis 


Additional  analysis  will  be  needed  to  identify  interventions  to  address 
current  and  future  safety  problems  for  which  few  or  no  previous  studies 
exist.  Safer  Skies  teams  relied  initially  on  a  limited  number  of  case  study 
analyses  to  identify  the  causes  of  accidents  and  incidents,  as  well  as  the 
interventions  that  could  prevent  them  in  the  future.  The  teams  compared 
the  results  of  these  case  studies  with  the  causes  and  interventions 
identified  by  previous  studies  to  determine  whether  they  are  consistent. 
For  example,  the  team  working  on  CFIT  in  commercial  aviation  completed 
detailed  event  sequences  for  10  accidents  and  found  that  the  causes 
identified  and  the  interventions  it  recommended  were  similar  to  those  of 
prior  studies.  Safer  Skies  teams  working  on  approach  and  landing 
accidents  in  commercial  aviation  and  weather-related  accidents  in  general 
aviation  also  compared  the  results  of  their  analyses  with  those  of  prior 
studies. 


Along  with  other  changes  as  the  Safer  Skies  initiative  has  evolved,  this 
approach  has  been  modified  as  teams  addressed  additional  safety 
problems.  For  example,  the  runway  incursion  analysis  team  expanded  its 
case  studies  to  include  incidents  because  there  were  so  few  fatal  accidents 
involving  runway  incursions.  Similarly,  the  analysis  team  now  working  on 
loss  of  control  in  commercial  aviation  has  selected  a  larger  number  of  case 
studies  because  this  safety  problem  has  not  been  the  subject  of  extensive 
prior  analysis. 


Effective 

Implementation  Is 
Critical  Next  Step  in 
Making  Progress 
Toward  the  Goals  Set 
for  Reducing  Fatal 
Accidents 


The  Safer  Skies  initiative  has  identified  the  major  safety  problems  to  be 
addressed,  has  made  progress  in  identifying  their  root  causes,  and  has 
developed  interventions  to  address  some  of  them.  Reducing  fatal  accidents 
depends  in  part  on  the  effective  implementation  of  these  interventions.  As 
discussed  in  chapter  one,  however,  many  of  these  safety  problems  are  long¬ 
standing  ones  that  have  persisted  in  spite  of  previous  studies  and 
recommendations.  In  addition,  FAA  has  not  consistently  followed  through 
on  implementing  safety  recommendations  in  the  past.  The  Safer  Skies 
initiative  does  not  yet  have  in  place  a  process  to  track  the  implementation 
of  these  interventions  that  is  sufficiently  detailed  and  covers  interventions 
chosen  to  improve  safety  in  commercial  aviation,  general  aviation,  and 
cabin  safety. 
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The  Success  of  Safer  Skies 
Interventions  Depends  on 
Effective  Implementation 


Reducing  the  fatal  accident  rate  in  commercial  aviation  and  the  number  of 
general  aviation  accidents  will  depend  in  part  on  effective  implementation 
of  the  interventions  chosen  by  the  Safer  Skies  teams.  Many  of  the  safety 
problems  that  the  initiative  addresses  are  long-standing  ones  that  have 
been  studied  extensively  in  the  past.  Actually  resolving  these  problems  has 
proven  difficult  in  the  past  and  remains  very  challenging.  Similar 
interventions  have  been  recommended,  but  the  desired  reductions  in  fatal 
accident  rates  have  not  been  achieved.  For  example,  extensive  prior 
studies  of  CFIT  and  approach  and  landing  accidents  in  commercial  aviation 
recommended  many  of  the  same  interventions  that  are  now  being 
implemented  by  the  Safer  Skies  commercial  aviation  steering  committee. 
Furthermore,  reaching  the  80-percent  goal  in  commercial  aviation  will 
depend  heavily  on  the  successful  implementation  of  interventions  to 
address  the  safety  problems  that  caused  the  most  fatal  accidents:  loss  of 
control,  CFIT,  and  approach  and  landing.  To  reach  the  goal  in  commercial 
aviation,  interventions  must  be  effectively  implemented  for  both  small 
commuter  aircraft  and  large  commercial  air  carriers.  Even  after  safety 
interventions  have  been  identified,  implementing  them  has  proven 
challenging. 
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As  DOT’S  Inspector  General  and  we  have  reported  previously,  FAA  does  not 
consistently  follow  through  on  implementing  safety  recommendations.  Our 
review  showed  that  FAA  usually  agreed  with  the  recommendations  on 
aviation  safety  made  by  GAO,  NTSB,  and  DOT’s  Inspector  General.  FAA 
had  implemented  64  percent  of  the  256  recommendations  that  we 
reviewed;  however,  FAA  had  not  completed  actions  to  implement  the 
remaining  36  percent  of  the  recommendations.2  We  found  that  FAA 
sometimes  did  not  establish  time  frames  for  implementing  the 
recommendations  or  did  not  meet  established  times  for  implementing 
them.  Similarly,  DOT’s  Inspector  General  found  that  of  the  23  near-term 
actions  FAA  planned  for  addressing  runway  incursions  in  its  1998  Action 
Plan,  15  had  not  been  completed  on  time.3  We  found  that  even  safety 
recommendations  that  received  specialized  attention,  intensive  follow-up, 
and  heightened  awareness  among  industry,  the  Congress,  and  the  public 
have  not  been  fully  implemented.  For  example,  NTSB  considered  runway 
incursions  so  serious  that  it  repeatedly  placed  this  safety  problem  on  its 
lists  of  critical  safety  recommendations  in  the  early  1990s.  Although  FAA 
concurred  with  NTSB’s  recommendations,  our  review  found  that  several  of 
the  corrective  actions  needed  had  not  been  implemented,  including  actions 
to  improve  (1)  visibility  at  airports;  (2)  runway  lighting,  signage,  and 
surface  markings;  and  (3)  radar  and  related  equipment  to  alert  air  traffic 
controllers  to  impending  runway  incursions.  FAA  developed  several  plans 
in  the  1990s  to  decrease  runway  incursions.  In  spite  of  these  programs,  the 
actual  number  of  runway  incursions  has  increased.  DOTs  Inspector 
General  noted  in  1999  that  the  number  of  runway  incursions  had  increased 
from  292  in  1997  to  325  in  1998,  in  part  because  FAA  had  not  set  aside  the 
funds  needed  to  support  the  initiatives  and  projects  in  the  runway 
incursion  action  plan.4  As  a  result,  FAA  has  made  limited  progress  in 
implementing  its  plan,  and  milestones  have  been  missed  and  extended. 
DOT’s  latest  performance  report  for  fiscal  year  1999  shows  continuing 
problems  in  this  area.  The  actual  number  of  runway  incursions  (321)  was 
19  percent  higher  than  the  goal  of  270  established  in  DOT’s  performance 
plan.5 


2 Aviation  Safety:  FAA  Generally  Agrees  With  but  Is  Slow  in  Implementing  Safety 
Recommendations  (GAO/RCED-96-193,  Sept.  23, 1996). 

2 Follow-up  Review  ofFAA’s  Runway  Safety  Program,  DOT.  FAA,  (AV-1999-1 14,  July  21 , 1999) . 

4 Federal  Aviation  Administration:  Aviation  Safety,  DOT  Inspector  General  Report  No.  A V- 
1999-069  (Statement  of  Alexis  M.  Stefani,  Deputy  Assistant  Inspector  General  for  Aviation 
U.S.  Department  of  Transportation  before  the  Subcommittee  on  Transportation  and  Related 
Agencies,  Committee  on  Appropriations,  U.S.  House  of  Representatives,  Mar.  10. 1999). 
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Industry  participants  in  the  Safer  Skies  initiative  have  voiced  concern  that 
some  interventions  may  not  be  implemented  promptly  or  at  all.  Some  of  the 
same  Safer  Skies  participants  questioned  whether  enough  resources  would 
be  available  to  complete  the  implementation  of  the  selected  interventions. 
Without  assurance  of  adequate  resources,  it  is  likely  that  the  choice  of 
interventions  by  Safer  Skies  teams  will  be  constrained  by  cost 
considerations  and  the  implementation  of  recommended  interventions  will 
be  incomplete.  Effective  implementation  will  also  depend  on  having  a 
process  for  tracking  the  implementation  of  interventions  to  be  carried  out 
by  all  Safer  Skies  participants,  including  FAA;  other  government  agencies; 
manufacturers;  airlines;  and  other  industry  participants. 


The  Steering  Committees 
Have  Not  Yet  Developed 
Effective  Processes  for 
Tracking  the 
Implementation  of 
Interventions 


FAA  and  the  aviation  industry  began  implementing  some  of  the  Safer  Skies 
safety  interventions  before  developing  a  systematic  way  of  tracking  the 
progress  being  made.  This  occurred  in  part  because  the  steering 
committees  incorporated  some  safety  initiatives  already  under  way  and 
endorsed  the  resulting  interventions  before  they  developed  a  systematic 
tracking  process.  In  addition,  Safer  Skies  teams  have  recommended  that  a 
few  high-priority  safety  initiatives  be  started  before  final  implementation 
reports  are  issued.  While  moving  forward  on  important  safety  initiatives 
makes  sense,  ensuring  their  successful  implementation  depends  on 
effective  tracking.  Interventions  have  been  implemented  in  both 
commercial  aviation  and  cabin  safety  with  no  tracking  process  in  place. 
The  general  aviation  steering  committee  is  moving  toward  approval  of 
interventions  for  CFIT  and  weather  but  has  not  yet  developed  a  tracking 
process.  Several  of  the  Safer  Skies  participants  we  interviewed  voiced 
some  concerns  about  whether  all  the  interventions  being  identified  would 
eventually  be  implemented,  given  FAA’s  past  problems  in  implementing 
recommended  safety  improvements. 


Tracking  Has  Been  Limited 
and  Not  Systematic 


In  its  December  1997  report,  the  congressionally  mandated  commission  on 
aviation  safety  recommended  that  FAA’s  and  the  industry’s  strategic  plan 
include  milestones  for  accomplishing  specific  tasks.  The  commission  noted 
that  the  plan  should  be  detailed  enough  that  milestones  for  accomplishing 
specific  tasks  can  be  readily  recognized  by  agency  management  and  the 
industry,  as  well  as  the  public.  In  addition,  the  commission  directed  FAA  to 


5DOT  FY  2001  Performance  Plan:  FY  1999  Performance  Report. 
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report  periodically  on  where  initiatives  stand,  why  any  delays  are 
occurring,  and  whether  and  why  changes  are  being  made  to  the  plan.  These 
recommendations  are  in  accordance  with  sound  internal  controls  for 
program  management. 

The  Safer  Skies  initiative,  which  was  announced  in  April  1998, 
implemented  a  number  of  interventions  without  first  developing  a  process 
for  tracking  their  progress.  In  some  cases,  these  were  interventions  that 
were  developed  by  teams  whose  work  was  incorporated  into  the  Safer 
Skies  effort.  In  commercial  aviation,  for  example,  FAA,  relying  on  the  work 
of  the  uncontained  engine  failure  team,  published  airworthiness  directives 
beginning  in  April  1999  to  require  more  extensive  inspections  of  aircraft 
engines.  The  commercial  aviation  team  working  on  CFIT  also  implemented 
several  interventions  in  or  before  September  1999.  These  included 
interventions  to  verify  the  operational  status  of  radar  equipment  to  provide 
minimum  safe  altitude  warnings  to  pilots  and  to  develop  a  template  for 
standard  operating  procedures  to  be  used  by  airlines  in  training  their  pilots 
in  techniques  to  avoid  CFIT  accidents. 
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In  September  1999,  the  commercial  aviation  steering  committee  recognized 
the  need  for  the  systematic  tracking  of  interventions  and  directed  a  work 
group  to  develop  a  proposal.  At  the  commercial  aviation  steering 
committee’s  meeting  in  January  2000,  the  work  group  presented  its 
proposal  for  a  Joint  Implementation  Measurement  Team.  The  team 
designed  the  tracking  process  to  provide  a  high-level  report  on  whether 
each  intervention  is  being  implemented  as  planned.  Specifically,  this  team’s 
responsibilities  will  include  tracking  whether  the  implementation  of 
approved  interventions  complies  with  the  implementation  plans  and  their 
milestones;  helping  to  predict  the  potential  effectiveness  of  the  proposed 
interventions;  and  identifying  ways  of  measuring  whether  the  intervention 
is  achieving  the  desired  risk  reduction.  The  team  will  also  provide  a  brief 
explanation  of  what  is  causing  noncompliance  with  the  plan  and  whether  a 
solution  has  been  found  to  resolve  the  problem.  As  conceived,  the  tracking 
report  is  to  be  a  high-level  progress  report  that  does  not  intrude  on  the 
internal  planning  of  the  organizations  responsible  for  carrying  out  the 
interventions.  The  tracking  report  thus  does  not  provide  detailed 
information  on  interim  and  long-term  milestones  or  identify  individuals 
responsible  for  implementing  the  plan  and  preparing  progress  reports  for 
the  tracking  committee.  Without  more  detailed  information  than  is 
currently  provided  in  the  proposed  tracking  report,  it  may  be  difficult  for 
the  steering  committee  to  assess  progress  in  implementing  interventions. 
For  example,  the  tracking  team’s  January  report  notes  that  FAA  has 
completed  a  plan  for  implementing  two  programs  critical  to  gaining  access 
to  safety  data6  and  that  other  industry  and  government  groups  have  plans  in 
development.  However,  the  tracking  report  provides  no  information  about 
the  milestones  established  by  FAA’s  plan  for  establishing  these  key 
programs,  both  of  which  have  experienced  delays  in  the  past.  After  we 
identified  concerns  about  the  tracking  system,  the  commercial  aviation 
steering  committee  agreed  that  improvements  are  needed,  and  it  is  working 
on  revisions.  A  draft  version  provided  for  our  review  in  June  2000  still 
lacked  key  information  about  major  commitments,  deliverables,  and 
milestones. 

Tracking  implementation  is  even  more  critical  for  the  more  complex 
initiatives  whose  success  depends  on  coordinated  efforts  by  both  FAA  and 
the  aviation  industry.  For  example,  successful  implementation  of  the 


'These  two  programs  are  the  Flight  Operational  Quality  Assurance  program  and  the  Aviation 
Safety  Action  Program.  Both  programs  facilitate  the  collection,  protection,  and  analysis  of 
safety  data  voluntarily  submitted  by  airlines  or  pilots. 
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highest-priority  intervention  to  prevent  CFIT  accidents  in  commercial 
aircraft— the  installation  of  enhanced  aircraft  navigational  equipment  to 
warn  pilots  of  impending  crashes — requires  coordination  among  many 
parties: 

•  FAA  must  certify  that  the  equipment  works,  issue  technical  standards 
for  manufacturers,  and  issue  a  final  regulation  to  require  that  the 
equipment  be  installed  on  new  and  existing  aircraft. 

•  Aircraft  manufacturers  need  to  make  the  equipment  standard  on  new 
aircraft  and  retrofit  it  in  older  aircraft. 

•  Air  carriers  need  to  incorporate  the  appropriate  procedures  for 
maintaining  and  using  this  equipment  into  maintenance  and  flight 
manuals  and  to  train  pilots  in  its  use. 

•  FAA  needs  to  update  its  guidance  to  its  inspectors  so  that  they  can 
ensure  that  air  carriers  properly  carry  out  their  responsibilities  for 
training,  maintenance,  and  use  of  the  equipment. 

Without  a  tracking  system  that  provides  more  detailed  information  on  the 
implementation  of  complex  interventions,  the  commercial  aviation  steering 
committee  will  not  have  the  information  needed  to  ensure  that  they  are 
fully  implemented  in  accordance  with  planned  milestones. 

The  implementation  of  interventions  to  improve  cabin  safely  has  also  not 
been  adequately  tracked.  The  cabin  safety  steering  committee,  which 
completed  the  development  of  passenger  education  materials  before  it 
disbanded  in  January  1999,  carried  out  most  of  its  interventions  with  no 
Safer  Skies  tracking  process  in  place.  However,  we  found  that  educational 
materials  related  to  passenger  interference  with  crew  had  not  been 
distributed  or  made  available  on  FAA’s  Web  page  as  of  April  2000. 
Furthermore,  according  to  a  member  of  the  cabin  safety  steering 
committee,  the  distribution  of  other  cabin  safety  brochures  was,  in  some 
instances,  never  completed.  The  absence  of  a  systematic  process  for 
tracking  Safer  Skies  interventions  may  have  contributed  to  inaccuracies  in 
reporting  on  the  status  of  cabin  safety  interventions.  Specifically,  the  DOT 
FY  2001  Performance  Plan  and  FY  1999  Performance  Report  states  that  all 
initiatives  relating  to  cabin  safety  were  completed  as  planned.  However, 
planned  actions  to  include  material  on  passenger  interference  with  crew 
had  not  been  completed  as  of  April  2000. 

Finally,  although  the  general  aviation  steering  committee  is  reviewing  draft 
implementation  team  reports  that  recommend  interventions  to  address 
CFIT  and  weather,  it  has  no  process  in  place  to  track  the  implementation  of 
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interventions  once  they  are  approved.  According  to  the  FAA  co-chairs  of 
the  general  aviation  steering  committee,  this  group  has  committed  to  track 
the  interventions  selected  but  has  not  yet  developed  a  process  for  doing 
that  and  plans  to  discuss  this  issue  at  a  future  meeting.  Without 
coordinated,  detailed  implementation  plans  that  assign  responsibilities, 
FAA  and  the  Safer  Skies  steering  committees  will  not  be  able  to  ensure  that 
all  parties  complete  their  portion  of  the  plan  and  that  implementation 
occurs  on  time.  In  addition,  as  part  of  the  Safer  Skies  process,  FAA  and  the 
general  aviation  community  identified  efforts  that  could  be  accomplished 
in  the  short  term  or  were  already  under  way  to  address  the  safety  areas  to 
be  addressed  by  the  initiative.  FAA  and  the  industry  implemented  a  number 
of  these  short-term  initiatives,  such  as  the  development  and  distribution  of 
various  safety  videos  and  training  aids.  However,  Safer  Skies  did  not  track 
the  implementation  of  these  interventions  or  evaluate  their  effectiveness. 


Conclusions  The  progress  made  by  the  initiative  to  date  has  resulted  in  the 

implementation  of  interventions  for  five  safety  problems — two  in 
commercial  aviation  and  three  in  cabin  safety.  However,  a  coordinated, 
centralized  method  of  tracking  will  be  necessary  to  ensure  full 
implementation  of  these  and  future  interventions.  In  the  past,  FAA  has 
developed  plans  to  make  safety  improvements  but  has  not  consistently 
implemented  them  successfully.  An  effective  tracking  system  would 
provide  for  identifying  the  individuals  or  entities  responsible  for 
implementation,  setting  milestones,  establishing  resource  estimates,  and 
preparing  progress  reports.  Without  a  systematic  tracking  mechanism, 
there  is  no  assurance  that  any  of  the  selected  interventions  will  be  fully 
implemented.  While  the  commercial  aviation  steering  committee  has 
developed  a  system  to  track  the  implementation  of  the  interventions  it 
approves,  this  system  is  not  sufficiently  detailed  to  ensure  their 
implementation.  The  general  aviation  steering  committee,  which  is  nearing 
final  approval  on  interventions  to  address  safety  problems  related  to 
weather  and  controlled  flight  into  terrain,  is  only  now  developing  a  tracking 
system  modeled  after  the  one  used  by  the  commercial  aviation  steering 
committee.  Finally,  nothing  comparable  has  been  developed  to  track 
interventions  recommended  by  the  cabin  safety  teams. 


Recommendations 


To  ensure  that  interventions  are  implemented  and  that  effective  and 
feasible  interventions  are  identified  in  the  future  for  issues  that  the 
initiative  has  yet  to  address,  we  recommend  that  the  Secretary  of 
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Transportation  direct  the  FAA  Administrator  to  advise  the  Safer  Skies 
steering  committees  to  take  the  following  actions: 

•  Develop  a  systematic  way  of  tracking  the  implementation  of 

interventions  approved  by  all  Safer  Skies  steering  committees.  This 
tracking  system  should  include  the  identification  of  responsibility  for 
implementation,  the  establishment  of  short-  and  long-term  milestones 
and  resource  estimates,  and  the  preparation  of  progress  reports.  The 
progress  reports  should  provide  information  on  the  detailed  steps  to  be 
taken  by  all  government  and  industry  participants  to  ensure  the 
successful  implementation  of  each  intervention.  Progress  reports 
should  highlight  and  explain  any  delays  in  meeting  the  milestones.  This 
system  should  be  shared  with  the  relevant  Safer  Skies  steering 
committees  and  FAA’s  focal  point  for  the  initiative  as  well  as  with  the 
team  that  recommended  the  intervention. 


Agency  Comments 


DOT  and  FAA  officials  concurred  with  our  recommendation  on  the  need  to 
track  the  implementation  of  interventions  to  achieve  results,  but  they 
disagreed  with  the  level  of  detail  we  advised.  The  officials  stated  that  the 
commercial  aviation  steering  committee’s  draft  revised  tracking  system 
provides  better  information  for  tracking  the  major  commitments  and 
deliverables.  The  expectation  is  that  more  detailed  implementation  plans 
will  be  maintained  within  each  implementing  organization.  The  officials  do 
not  believe  that  it  is  realistic  for  steering  committees  to  review  the  details 
of  every  organization’s  action  plan.  They  also  noted  that  the  general 
aviation  steering  committee  is  developing  a  tracking  system  similar  to  that 
used  to  track  commercial  aviation  interventions.  We  agree  that  the  Safer 
Skies  initiative  has  taken  steps  to  improve  its  tracking  system  for 
commercial  aviation  and  to  work  toward  the  development  of  a  similar 
system  for  general  aviation.  However,  the  revised  tracking  system  provided 
for  our  review  in  June  2000  did  not  clearly  identify  and  include  time  frames 
for  major  commitments  and  deliverables  for  each  of  the  interventions 
approved  by  the  commercial  aviation  steering  committee.  We  agree  that 
individual  FAA  and  industry  organizations  responsible  for  implementing 
Safer  Skies  interventions  would  logically  have  far  more  detailed  systems 
for  tracking  implementation  than  the  steering  committees.  However, 
without  a  reliable  tracking  system  in  place  that  contains  basic  information 
on  major  deliverables,  responsibilities,  and  time  frames,  FAA  and  Safer 
Skies  will  not  be  in  a  position  to  ensure  that  recommended  interventions 
are  implemented  to  improve  aviation  safety. 
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DOT  and  FAA  officials  disagreed  with  our  recommendation  that  FAA  and 
the  Safer  Skies  steering  committees  should  analyze  a  sample  of  safety 
problems  that  were  not  studied  previously.  The  officials  presented 
information  that  showed  some  Safer  Skies’  work  groups  were  using  or 
would  be  using  a  sample  of  previously  unexamined  safety  problems  in  their 
work.  For  this  reason,  we  withdrew  the  recommendation. 
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Of  the  five  Safer  Skies  teams  that  have  begun  implementing  interventions, 
only  one  has  developed  a  performance  measure  to  evaluate  whether  the 
interventions  it  has  selected  are  helping  to  reduce  the  safety  problems  that 
cause  fatal  accidents  and  are  worth  what  they  cost.  Such  evaluations 
depend  on  performance  measures  that  serve  as  the  yardsticks  for 
measuring  progress  toward  program  goals.  The  initiative’s  ultimate  goal  is 
saving  lives  by  reducing  fatal  accidents.  Federal  law  requires  that  federal 
departments  evaluate  the  effectiveness  of  the  program  activities  for  which 
they  request  funding.  FAA  will  evaluate  progress  toward  its  broad  goals  for 
aviation  safety  using  performance  measures  based  on  reducing  the  fatal 
accident  rate  for  commercial  aviation  and  the  number  of  fatal  accidents  in 
general  aviation.  However,  additional  performance  measures  will  be 
needed  for  evaluating  the  effectiveness  of  the  interventions  selected  by  the 
teams  working  on  each  of  the  safety  problems.  Most  teams  are  still 
analyzing  data  on  safety  problems  and  selecting  safety  interventions  and 
thus  have  not  yet  determined  how  to  evaluate  the  effectiveness  of 
interventions  selected.  Although  teams  working  on  5  of  the  16  safety 
problems  have  recommended  interventions  that  are  being  implemented, 
only  one  of  these  teams  developed  an  adequate  performance  measure 
before  its  interventions  were  implemented. 


Federal  Law  Requires 
the  Development  of 
Performance  Measures 
as  Part  of  the  Budget 
Process 


To  ensure  that  programs  achieve  their  objectives  and  that  funds  are 
expended  wisely,  federal  law  requires  that  each  department  develop 
performance  measures  as  part  of  its  budget  request.  Performance 
measures  are  the  yardsticks  used  to  evaluate  the  effectiveness  of  the 
activities  undertaken  as  part  of  federal  programs.  The  initiative  plans  to 
develop  performance  measures  to  evaluate  the  effectiveness  of  the 
interventions  it  recommends  to  save  lives  by  addressing  the  safety 
problems  that  cause  fatal  accidents.  However,  developing  good 
performance  measures  can  be  difficult.  While  it  is  useful  to  establish  a 
baseline  of  information  about  past  fatal  accidents,  they  occur  too  rarely  to 
serve  as  performance  measures  to  evaluate  the  effectiveness  of 
interventions.  Years  may  elapse  between  specific  types  of  fatal  accidents, 
such  as  uncontained  engine  failure,  making  it  difficult  to  see  trends  or 
evaluate  the  effectiveness  of  interventions.  Instead,  the  initiative  must 
develop  performance  measures  based  on  events  that  occur  more  frequently 
and  that  can  be  linked  closely  to  interventions. 


A  congressional  mandate  exists  for  the  measurement  and  evaluation  of  all 
federal  programs.  Performance  measurement  is  a  central  premise  of  the 
Government  Performance  and  Results  Act  of  1993  (Results  Act).  This  act 
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requires  annual  performance  plans  to  cover  each  program  activity  set  out 
in  a  federal  agency’s  budget.  Among  other  requirements,  performance  plans 
are  to  (1)  establish  performance  indicators  to  be  used  in  measuring  or 
assessing  the  outcomes  of  each  program  activity,  (2)  determine  how  to 
compare  actual  results  with  the  performance  goals,  and  (3)  describe  the 
means  to  verify  and  validate  information  used  to  report  on  performance.  In 
accordance  with  this  law,  DOT  develops  annual  plans  that  include 
performance  measures  for  specific  programs  and  activities.  Agencies  under 
DOT,  such  as  FAA,  develop  more  detailed  plans  and  performance  measures 
for  each  program  activity. 

Because  of  its  impact  on  FAA’s  programmatic  and  budgeting  activities,  the 
Safer  Skies  initiative  falls  under  the  Results  Act’s  requirement  to  evaluate 
progr  am  performance.  Moreover,  it  was  developed  in  response  to  the 
National  Civil  Aviation  Review  Commission’s  report,  which  specifically 
directed  FAA  and  the  aviation  industry  to  establish  performance  measures 
and  milestones  to  assess  the  initiative’s  progress  in  meeting  safety  goals,  to 
review  priorities  periodically,  and  to  monitor  progress.  The  Safer  Skies 
initiative  incorporates  the  idea  of  establishing  performance  measures  to 
evaluate  progress  toward  safety  goals.  As  a  result,  the  Safer  Skies  teams 
that  recommend  interventions  are  tasked  with  developing  the  performance 
measures  for  those  interventions  approved  by  the  steering  committees. 

For  a  performance  measure  to  be  useful,  a  baseline  must  be  established 
against  which  to  measure  the  effect  of  the  intervention.  Good  evaluation 
criteria  include  (1)  definitions  of  baseline  information  on  the  extent  of  the 
safety  problem  over  a  particular  period  prior  to  the  implementation  of  the 
intervention  and  (2)  timeframes  for  evaluating  changes  using  the 
performance  measure.  Goals  and  time  frames  must  also  be  established  to 
determine  what  the  program  is  expected  to  achieve  and  by  when.  For  the 
initiative,  appropriate  baseline  information  includes  both  the  total  number 
of  fatal  accidents  and  the  number  of  fatal  accidents  caused  by  each  safety 
problem  within  each  type  of  aviation  operation  (i.e.,  commercial  aviation 
and  general  aviation). 

Good  performance  measures  have  several  key  features:  the  event  to  be 
measured  (e.g.,  a  runway  incursion)  or  desired  outcome  (a  reduction  in  the 
number  of  runway  incursions)  is  measurable;  data  on  the  event  are  or 
could  be  collected;  and  the  event  occurs  with  sufficient  frequency  between 
evaluations  for  progress  to  be  measurable.  The  performance  measures 
under  development  to  evaluate  Safer  Skies’  initiatives  can  be  assessed 
against  these  criteria. 
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Determining  the 
Effectiveness  of  Safer 
Skies’  Initiatives  Will 
Require  the 
Development  of 
Additional 

Performance  Measures 


Determining  the  effectiveness  of  Safer  Skies  interventions  will  require  the 
development  of  performance  measures  other  than  the  overall  goals  set  for 
commercial  and  general  aviation.  Fatal  aviation  accidents  occur  so 
infrequently  that  their  usefulness  is  limited  as  a  measure  of  the  success  of 
Safer  Skies'  interventions.  This  is  especially  true  for  commercial  aviation, 
which  had  a  total  of  85  fatal  accidents  in  the  United  States  from  1988 
through  1997.  The  fact  that  a  particular  type  of  accident  has  not  occurred 
for  several  years  does  not  mean  that  the  underlying  safety  problem  has 
been  successfully  addressed.  Furthermore,  for  several  reasons  it  may  be 
difficult  or  impossible  to  match  a  specific  implementation  plan  to  a 
numerical  reduction  in  fatal  accidents  overall  or  attributable  to  a  specific 
safety  problem.  For  example,  in  general  aviation  the  lack  of  detail  in 
accident  reporting  makes  it  difficult  to  determine  specific  accident  causes; 
the  lack  of  pilot  profiles  makes  it  difficult  to  evaluate  the  effectiveness  of 
pilot  training  strategies;  and  it  is  hard  to  predict  how  many  aircraft  owners 
will  install  new  safety  equipment  in  the  future.  Thus,  to  determine  to  what 
extent  an  intervention  is  reducing  fatal  accidents  attributed  to  a  specific 
safety  problem,  teams  will  need  to  develop  additional  performance 
measures.  The  commercial  aviation  steering  committee  recognized  early 
on  this  need  to  develop  interim  measures  to  evaluate  the  unique  effect  of 
individual  interventions. 


Even  if  a  team  identifies  suitable  performance  measures  for  a  specific 
safety  problem,  it  may  be  difficult  to  determine  whether  a  particular 
intervention,  cluster  of  interventions,  or  other  outside  factors  influenced 
changes  in  the  performance  being  measured.  This  is  especially  true  for 
situations  in  which  teams  choose  numerous  interventions  to  address  a 
safety  problem.  While  the  uncontained  engine  failure  team  developed  a 
single  primary  intervention,  the  team  working  on  CFIT  in  commercial 
aviation  has  already  initiated  several  interventions  and  is  contemplating 
dozens  more.  Similarly,  the  general  aviation  team  working  on  weather 
recommended  17  interventions.  Without  some  way  to  independently 
evaluate  the  effectiveness  of  individual  interventions  or  clusters  of 
interventions,  the  initiative  will  have  little  way  of  knowing  whether 
particular  interventions  save  lives  and  are  thus  worth  the  time  or  money 
being  expended  on  them. 

In  developing  performance  measures,  one  option  involves  using  the 
precursors  to  accidents  as  proxies  for  the  likelihood  of  fatal  accidents. 
Precursors  are  events  that,  although  they  typically  precede  a  particular 
type  of  fatal  accident,  often  occur  without  culminating  in  a  crash.  For 
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example,  approach  and  landing  accidents  are  almost  always  preceded  by 
unstable  approaches  to  the  airport,  but  many  unstable  approaches  may 
culminate  in  a  hard  or  late  landing  that  does  not  result  in  injuries  or  a  crash. 
Performance  measures  based  on  precursors  have  been  developed  to 
evaluate  initiatives  for  one  of  the  safety  problems  the  initiative  is 
addressing,  uncontained  engine  failure.  The  success  of  this  approach 
depends  on  identifying  appropriate  accident  precursors  that  can  serve  as 
proxies  for  the  specific  safety  problem  the  team  is  addressing.  Precursors 
are  most  useful  when  they  follow  the  criteria  for  good  performance 
measures:  they  are  measurable,  relevant  data  on  them  are  available,  and 
they  occur  with  sufficient  frequency. 


Most  Safer  Skies’ 
Interventions  Are 
Being  Implemented 
Without  Determining 
How  to  Evaluate  Their 
Effectiveness 


Of  the  16  Safer  Skies  teams,  8  have  recommended  safety  interventions  for 
implementation;  interventions  from  5  of  these  teams  have  been  or  are  being 
implemented;  but  only  one  has  developed  a  performance  measure  that  can 
show  whether  the  intervention  is  effective  at  saving  lives.  Most  Safer  Skies 
teams  are  still  analyzing  data  on  safety  problems  and  selecting 
interventions  and  have  not  yet  determined  how  to  evaluate  the 
effectiveness  of  interventions  selected.  Of  the  five  teams  whose 
recommendations  are  being  implemented,  three  have  developed  some 
performance  measures.  Only  the  uncontained  engine  failure  team  has 
developed  two  quantifiable  performance  measures  that  are  based  on 
accident  precursors.  In  contrast,  the  general  aviation  teams  working  on 
CFIT  and  weather  developed  some  general  performance  measures  for 
reducing  accidents  resulting  from  these  safety  problems  but  did  not 
quantify  these  measures.  No  performance  measures  were  developed  to 
evaluate  the  educational  interventions  implemented  to  address  the  four 
cabin  safety  problems.  Finally,  the  team  working  on  CFIT  accidents  in 
commercial  aviation  has  implemented  one  intervention  in  advance  of  the 
team’s  final  report.  While  this  team  has  not  yet  developed  a  performance 
measure  for  this  intervention,  it  is  considering  using  an  accident  precursor. 
Performance  measures  based  on  accident  precursors  have  potential  for  use 
in  evaluating  the  effectiveness  of  additional  interventions  being  considered 
to  address  CFIT  and  other  safety  problems.  FAA  does  not  presently  collect 
data  on  some  accident  precursors  that  could  be  used  to  evaluate  the 
effectiveness  of  Safer  Skies  interventions  and  faces  significant  barriers  to 
collecting  such  data. 
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The  Uncontained  Engine 
Failure  Team  Has  Chosen 
Two  Accident  Precursors  as 
Performance  Measures 


The  Safer  Skies  team  working  on  uncontained  engine  failure  chose  two 
accident  precursors  as  performance  measures  for  evaluating  the 
effectiveness  of  the  intervention  it  recommended:  more  extensive  engine 
inspections.  Because  uncontained  engine  failure  caused  just  two  fatal 
accidents  in  the  United  States  in  1988-97,  fatal  accidents  are  too  infrequent 
to  serve  as  a  performance  measure.  But  well-established  trend  data  show 
that  the  safety  problem  occurs  much  more  frequently,  resulting  not  in  fatal 
accidents  but  in  incidents  with  severe  or  serious  consequences  on  an 
average  of  about  1.5  times  a  year.1  The  team  chose  the  rate  of  these 
incidents  as  the  primary  performance  measure  for  its  recommended 
intervention.  The  team  also  chose  another  accident  precursor  as  a  second 
performance  measure:  the  number  of  cracks  detected  in  engine  disks  when 
engines  are  overhauled.2  Data  analysis  identified  cracked  disks  as  the 
primary  cause  of  uncontained  engine  failure.  According  to  staff  at  FAA’s 
Engine  and  Propeller  Directorate,  each  crack  detected  during  inspections 
probably  avoids  an  uncontained  engine  failure  that  could  have  had  severe 
or  serious  consequences. 


Both  accident  precursors  chosen — the  rate  of  uncontained  engine  failure 
with  severe  or  serious  consequences  and  the  detection  of  cracks  in  engine 
disks — have  some  of  the  attributes  of  a  good  performance  measure.  Both 
can  be  counted,  and  reporting  mechanisms  are  in  place  for  collecting  the 
key  data  needed  for  both  measures.  Hence,  it  will  be  possible  to  evaluate 
whether  the  more  extensive  engine  inspections  lead  to  the  detection  of 
more  cracks  and  fewer  instances  of  uncontained  engine  failure  with  severe 
or  serious  consequences.  However,  good  performance  measures  track 
events  that  occur  often  enough  between  evaluations  to  show  whether 
progress  is  being  made.  Uncontained  engine  failure  with  severe  or  serious 
consequences  occurs  from  one  to  three  times  a  year,  according  to  data 
from  1992-98,  while  cracks  in  engine  disks  are  likely  to  be  discovered  about 
once  in  25,000  inspections,  according  to  staff  at  FAA’s  Engine  and  Propeller 


'Severe  consequences  (level  4  events)  include  fatal  or  serious  injury,  loss  of  the  aircraft  hull, 
and  forced  landing  of  the  aircraft.  Serious  consequences  (level  3  events)  include  substantial 
damage  to  the  aircraft  or  an  unrelated  system,  uncontrolled  fire,  rapid  cabin 
depressurization,  temporary  or  permanent  inability  to  climb  or  fly  the  aircraft  1,000  feet 
above  terrain,  and  temporary  or  permanent  impairment  of  the  aircraft’s  controllability. 

2Disks  are  heavy,  high-speed  rotating  parts  inside  an  engine  with  attached  fan  blades  that 
produce  thrust.  Undetected  manufacturing  flaws  or  contaminants  can  undermine  a  disk’s 
structural  integrity,  allowing  a  crack  to  occur.  If  a  crack  causes  a  disk  to  fail  and  break  apart, 
fast-moving  fragments  of  the  disk  can  disable  or  damage  the  airplane  and  may  have 
catastrophic  results. 
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Directorate.  Hence,  2  to  5  years  may  elapse  before  the  effectiveness  of  the 
more  extensive  engine  inspections  can  be  judged.  Nonetheless,  tracking 
both  measures  should  provide  sufficient  data  for  reasonable  interim  and 
final  performance  measures,  and  the  enhanced  inspections  provide  an 
opportunity  to  avert  potentially  catastrophic  accidents. 

The  uncontained  engine  failure  team  established  much  of  the  information 
needed  to  use  its  performance  measures  to  evaluate  the  effectiveness  of 
enhanced  engine  inspections.  During  our  review,  we  worked  with  FAA  staff 
on  the  team  to  develop  additional  information  to  provide  a  more  complete 
context  for  how  that  intervention  relates  to  the  overall  Safer  Skies  effort 
and  to  the  fatal  accident  rate  in  commercial  aviation.  We  then  developed  a 
template  for  this  information  that  can  serve  as  a  model  for  other  Safer 
Skies  implementation  teams.  (See  table  8.)  The  template  displays  the  data 
critical  for  understanding  the  extent  of  the  safety  problem  and  the  baseline 
for  measuring  progress  in  addressing  it,  including  the  frequency  of  the 
problem’s  occurrence  in  1988-97  and  projections  of  its  occurrence  with  and 
without  the  recommended  intervention  by  2007,  the  target  year  for  Safer 
Skies  to  achieve  an  80-percent  reduction  in  the  overall  fatal  accident  rate. 
The  template  reflects  the  team’s  goal  of  reducing  the  rate  and  projected 
number  of  uncontained  engine  failures  with  severe  or  serious 
consequences  by  50  percent  by  2007. 


Table  8:  Baselines  and  Goals  for  Reducing  the  Occurrence  of  Uncontained  Engine  Failure  in  Commercial  Aviation 


Baseline  information  on  uncontained  engine  failure 

Cause  of  fatal  accident 

Baseline  (1988-97 

Annual  average 

Projected  for  2007 
without  intervention  d 

80-percent 

reduction 

All  causes 

85 

6 

9 

2 

Uncontained  engine  failure 

2 

c 

c 

c 

Baseline  information  on  intervention  and  performance  measures 

Performance  measure 

Baseline 

Annual  average 

Projected  for  2007 
without  intervention  e 

Projected  for  2007 
with  intervention 

Rate  of  detection  of  cracks 
(estimated) 

a 

a 

a 

1  per  25,000 
inspections 

Rate  of  uncontained  engine  failure 
events  with  severe  or  serious 

0.9  per  10  million 
departures6 

0.9  per  10  million 
departures 

0.9  per  10  million 
departures 

0.45  per  10  million 
departures 

consequences 

Number  of  uncontained  engine 
failure  events  with  severe  and 

nb 

1.5 

2 

1 

serious  consequences 
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“Past  data  on  the  number  of  disk  inspections  are  not  available.  Thus,  data  on  the  rate  of 
detection  of  cracks  under  the  prior  requirements  for  engine  inspections  are  not  available. 

Established  using  available  baseline  data  from  1992-98. 

cBecause  uncontained  engine  failure  occurs  so  infrequently,  both  the  annual  average  of  0.2 
and  any  projections  are  unreliable. 

riOur  projection  uses  FAA’s  projected  annual  growth  rates  of  4.0  percent  for  large  air  carriers 
and  3.0  percent  for  commuter  air  carriers. 

eFAA’s  projection  is  based  on  Boeing’s  data  showing  growth  in  annual  aircraft  departures 
from  16  million  in  1998  to  22  million  in  2007. 

'The  annual  average  is  for  the  baseline  years  chosen  by  FAA  and  the  Safer  Skies  steering 
committees  in  establishing  their  goal  for  commercial  aviation:  1994-96. 

Source:  GAO’s  analysis  of  data  from  Safer  Skies,  FAA,  and  NTSB. 


General  Aviation  Teams  Did 
Not  Develop  Quantified, 
Specific  Performance 
Measures 


The  general  aviation  implementation  teams  for  CFIT  and  weather  have 
completed  their  draft  reports  but  did  not  develop  quantified,  specific 
performance  measures  to  evaluate  the  effectiveness  of  the  interventions 
they  recommended.  The  general  aviation  CFIT  team  recommended  5 
interventions  subdivided  into  22  distinct  subinterventions.  None  of  the  22 
subinterventions  included  specific,  quantified  performance  measures.  For 
example,  the  CFIT  team  recommended  developing  criteria  for 
standardizing  the  marking  of  wires,  towers,  and  support  structures  to  help 
decrease  the  number  of  CFIT  accidents  that  occur  when  pilots  of  low-flying 
aircraft,  such  as  helicopters  and  small  planes,  fly  into  these  obstacles.  As 
one  measure  of  effectiveness,  the  team  chose  a  decrease  in  the  number  of 
CFIT  accidents  involving  wires  or  towers.  However,  the  team  did  not 
provide  any  baseline  information  about  the  number  of  past  CFIT  accidents 
that  involved  wires  or  towers  or  the  types  of  aircraft  involved.  To  the  extent 
that  such  baseline  information  is  available,  it  provides  a  yardstick  against 
which  to  measure  progress  in  reducing  these  accidents.  Furthermore,  the 
team  did  not  provide  any  specific  interim  or  long-term  accident  reduction 
goals  for  the  number  of  accidents  or  the  percentage  of  the  fleet  affected. 
Without  such  information,  it  will  be  impossible  to  determine  whether  or  by 
how  much  CFIT  accidents  involving  wires  or  towers  have  decreased.  Other 
performance  measures  for  general  aviation  CFIT  initiatives  share  this  lack 
of  quantification  and  specificity.  Without  baseline  information  on  the 
occurrence  of  the  problem  prior  to  the  implementation  of  the  intervention 
and  specific  quantified  goals,  it  will  be  impossible  to  evaluate  the 
effectiveness  of  the  interventions  implemented. 
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The  general  aviation  team  working  on  weather  experienced  similar 
problems  in  setting  performance  measures  for  its  interventions.  The  team  s 
final  report  recommended  17  interventions  subdivided  into  49  distinct 
subinterventions.  Of  the  49  subinterventions,  only  1  included  a  quantified, 
specific  performance  measure.  The  rest  had  either  no  performance 
measures  or  performance  measures  that  were  not  quantified  or  specific. 
Some  of  the  interventions  for  which  no  performance  measures  were 
established  involve  research  that  is  still  ongoing  to  develop  the  technology 
suggested  in  the  intervention.  For  example,  NASA  has  the  lead  in 
developing  equipment  to  sense  turbulence  and  warn  flight  crews  so  that 
they  can  avoid  or  reduce  the  dangers  associated  with  turbulence.  Because 
research  on  this  technology  is  preliminary,  the  performance  measures  are 
described  broadly  as  reducing  fatalities  and  injuries.  It  is  likely  too  early  to 
establish  performance  measures  for  these  interventions.  The  performance 
measures  included  for  many  other  subinterventions  were  too  broad  to 
allow  actual  evaluation  of  their  effectiveness.  The  performance  measure 
for  most  of  these  was  a  “decrease  in  the  number  of  weather-related 
accidents.”  These  performance  measures  are  neither  quantified  nor  linked 
in  any  specific  way  to  the  interventions,  which  makes  it  impossible  to 
determine  what  portion  of  the  reduction,  if  any,  is  attributable  to  individual 
interventions  or  clusters  of  interventions.  Of  the  performance  measures 
developed,  several  measure  progress  in  implementing  training 
interventions,  rather  than  the  effectiveness  of  the  training  in  reducing 
safety  threats.  For  example,  one  intervention  involves  training  Flight 
Service  Station  specialists  and  supervisors  on  in-depth  weather  analysis 
and  interpretation  to  improve  the  weather  briefings  given  to  general 
aviation  pilots.  The  associated  performance  measure  involves  training  all 
of  these  FAA  staff  by  2002,  rather  than  measuring  the  effectiveness  of  that 
training.  In  other  cases,  the  team  did  not  include  a  performance  measure 
when  one  could  have  been  developed.  For  example,  one  intervention 
involves  conducting  a  refresher  clinic  for  flight  instructors  to  update  them 
about  current  weather  information  and  provide  appropriate  training 
materials  for  them  to  use  with  general  aviation  pilots.  No  performance  goal 
was  specified  for  this  intervention.  To  measure  how  well  this  intervention 
has  been  implemented,  it  is  possible  to  determine  the  number  of  flight 
instructors,  to  establish  a  goal  for  how  many  attend  this  training  each  year, 
and  to  have  them  provide  information  on  how  many  pilots  they 
subsequently  train  using  the  information.  To  determine  whether  the 
intervention  is  effective,  the  pilots  who  receive  the  training  could  later  be 
surveyed  to  determine  whether  they  had  used  the  weather  information 
provided  or  their  safety  records  could  be  compared  with  the  records  of 
pilots  who  did  not  have  the  training.  The  link  between  accident  reduction 
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and  such  training  is  more  tenuous  than  the  link  between  crack  detection 
and  the  prevention  of  uncontained  engine  failure,  but  it  is  possible  to  gain 
at  least  some  information  about  the  effectiveness  of  the  training.  Without 
such  feedback,  it  is  difficult  to  determine  whether  the  training  is  effective 
and  should  be  continued. 

Without  more  specific  baseline  information  on  these  performance 
measures  prior  to  the  implementation  of  the  interventions  and  interim  and 
long-term  goals  for  progress,  the  initiative  will  not  be  able  to  evaluate  the 
impact  of  these  interventions.  In  responding  to  our  draft  report,  FAA  noted 
that  the  implementation  teams  for  CFIT  and  weather  relied  on  the 
expertise  of  team  members,  following  analysis  of  the  root  causes  of 
accidents,  to  determine  the  probable  effectiveness  of  the  interventions. 
Safer  Skies  analysis  and  implementation  reports  described  problems  with 
the  quantity,  quality,  and  type  of  data  currently  available  about  general 
aviation.  These  problems  include  shortcomings  in  the  data  for  the  types 
and  numbers  of  operations  and  in  the  level  of  detail  of  the  actual  accident 
investigations.  FAA  concluded  that  the  problems  with  general  aviation  data 
make  it  difficult  to  measure  the  effectiveness  of  individual  intervention 
strategies  by  the  traditional  approach  of  how  they  affect  accident  rates. 
While  we  acknowledge  the  need  to  improve  general  aviation  data,  we  also 
believe  that  such  data  can  provide  some  indication  of  the  relative  frequency 
and  importance  of  the  causes  of  fatal  accidents.  Such  information  is  also 
important  for  making  decisions  about  which  interventions  to  fund  and 
expedite,  considering  their  potential  effectiveness  and  the  number  of  fatal 
accidents  that  their  use  might  prevent.  While  it  may  not  be  possible  to 
develop  quantitative  performance  measures  for  all  interventions  proposed 
by  the  implementation  teams,  good  performance  measures  depend  on 
having  measurable  events,  a  way  to  collect  data  on  those  events,  and  an 
event  that  occurs  with  sufficient  frequency  between  evaluations  for 
progress  to  be  measurable.  The  performance  measures  for  both  general 
aviation  weather  and  CFIT  could  be  improved  where  possible  by 
identifying  and  quantifying  baseline  information,  ensuring  that  a  means 
exists  for  collecting  data  on  the  performance  measure,  and  setting  interim 
and  long-term  goals  against  which  to  measure  progress  in  implementing  the 
intervention. 
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The  Safer  Skies  Initiative 
Did  Not  Develop  a  Strategy 
for  Evaluating  Cabin  Safety 
Interventions 


The  Safer  Skies  cabin  safety  steering  committee  completed  work  on  four 
safety  problems  and  implemented  most  interventions  without  developing 
any  strategy  for  evaluating  the  interventions.  Although  the  steering 
committee  completed  its  work  in  January  1999,  it  did  not  develop 
performance  measures  for  the  interventions  it  selected.  While  the 
initiative  s  broad  goal  is  reducing  the  fatal  accident  rate,  the  broad  goal  for 
cabin  safety  is  educating  the  flying  public  about  four  areas:  passenger 
interference  with  flight  crews,  passenger  use  of  seat  belts,  child  restraint 
systems,  and  carry-on  baggage.  The  steering  committee  distributed 
brochures  about  carry-on  baggage  and  the  importance  of  child  restraint 
systems  and  worked  with  air  carriers  to  develop  additional  cabin 
announcements  to  remind  passengers  to  use  their  seat  belts.  The  team  did 
not,  however,  set  up  any  evaluation  to  show  whether  the  public’s 
knowledge  about  these  issues  improved  as  a  result  of  these  interventions 
and  whether  that  improved  knowledge  results  in  fewer  fatalities. 


While  useful  performance  measures  could  be  defined  in  each  of  the  four 
cabin  safety  areas,  the  steering  committee  did  not  develop  a  strategy  for 
evaluating  the  impact  of  its  educational  initiatives.  For  example,  the 
steering  committee  did  not  plan  or  track  the  distribution  of  the  flyers  it 
issued  about  carry-on  baggage  or  child  restraint  systems,  and  it  developed 
no  performance  measures  for  evaluating  the  effectiveness  of  these 
initiatives  to  educate  the  public.  Furthermore,  FAA  does  not  have  a 
mechanism  for  consistently  collecting  data  about  any  of  these  areas. 
Airlines  are  required  to  report  information  related  to  cabin  safety  only  if 
something  happens  in  the  cabin  that  results  in  serious  injuries  or  death.  As 
a  consequence,  the  agency  does  not  have  baseline  data  for  measuring 
improvements  that  may  result  from  its  initiatives.  Thus,  the  Safer  Skies 
initiative  has  no  way  of  measuring  the  effectiveness  of  its  educational 
efforts  in  the  cabin  safety  area. 


Precursors  of  Accidents 
Have  Potential  for  Use  as 
Performance  Measures  in 
Other  Safer  Skies  Areas 


Precursors  to  accidents  have  the  potential  for  use  as  performance 
measures  for  evaluating  interventions  to  address  at  least  three  other  Safer 
Skies  safety  problems:  CFIT,  runway  incursions,  and  approach  and  landing. 
Precursors  are  needed  because  fatal  aviation  accidents  caused  by  all  three 
safety  problems  occur  rarely.  The  precursors  for  each  safety  problem  have 
at  least  some  of  the  attributes  of  good  performance  measures. 
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The  Safer  Skies  team  working  on  CFIT  accidents  in  commercial  aviation  is 
considering  using  an  accident  precursor  to  evaluate  the  effectiveness  of 
one  of  its  interventions:  the  installation  of  enhanced  navigational 
equipment  on  aircraft  that  sounds  alerts  to  warn  pilots  of  impending 
crashes.  The  equipment  tracks  data  on  the  frequency  of  the  alerts  and  the 
situations  in  which  they  occur.  Although  these  data  are  not  currently 
collected  by  FAA,  they  could  be  used  to  develop  a  performance  measure 
based  on  the  alerts  sounded  as  precursors  to  CFIT  accidents.  The 
performance  measure  of  alerts  sounded  could  indicate  the  number  of 
dangerous  situations  avoided. 

Alerts  sounded  by  this  navigational  equipment  have  several  features  of  a 
good  performance  measure.  First,  the  alerts  can  be  measured.  Second,  the 
equipment  itself  tracks  such  warnings.  Finally,  the  alerts  are  sounded  with 
sufficient  frequency  to  be  useful  as  a  performance  measure.  According  to 
the  manufacturer,  enhanced  navigational  equipment  was  installed  in  over 
4,000  aircraft  from  March  1996  through  December  1999.  In  14  instances,  the 
alerts  enabled  pilots  to  recover  from  impending  crashes. 

Runway  incursion  incidents  that  do  not  result  in  accidents  provide  another 
useful  performance  measure  and  are  being  used  as  such  by  FAA.  From  1988 
through  1997,  2,345  runway  incursions  resulted  in  five  fatal  accidents  and 
59  fatalities  in  the  United  States.3  However,  runway  incursions  have  the 
potential  to  cause  much  greater  numbers  of  fatalities;  the  collision  of  two 
large  aircraft  on  the  ground  in  the  Canary  Islands  in  1977  resulted  from  a 
runway  incursion  and  took  more  than  580  lives.  Because  runway  incursion 
incidents  are  increasing  in  the  United  States  and  have  the  potential  to  lead 
to  fatal  accidents,  FAA’s  Performance  Plan  for  FY  2000  has  used  these 
incidents  to  establish  a  performance  measure  for  a  series  of  safety 
recommendations  designed  to  reduce  accidents  caused  by  runway 
incursions.  The  Safer  Skies  team  addressing  runway  incursions  has  not  yet 
identified  interventions,  but  FAA’s  ongoing  work  offers  some  useful 
performance  measures  for  measuring  progress  in  addressing  this  safety 
problem. 


3Five  runway  incursions  involving  commercial  aircraft  occurred  in  the  United  States  during 
this  period.  In  classifying  fatal  accidents,  the  commercial  aviation  steering  committee 
classified  one  of  these  accidents  as  an  approach  and  landing  accident  for  purposes  of 
analysis  because  one  plane  landed  on  top  of  another  plane. 
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Runway  incursion  incidents  have  all  three  features  of  a  good  performance 
measure.  First,  the  incidents  can  be  counted.  Second,  the  data  can  be 
collected  because  FAA  already  has  a  mechanism  for  reporting  runway 
incursions.4  Moreover,  FAA  has  collected  data  on  them  for  years,  and 
therefore  has  historical  data  that  can  be  used  to  establish  baselines  against 
which  the  effectiveness  of  interventions  intended  to  reduce  runway 
incursions  can  be  measured.  For  example,  one  intervention  now  in  use  by 
FAA  involves  deploying  action  teams  to  airports  that  have  experienced  high 
numbers  of  runway  incursion  incidents  to  determine  the  causes  and 
develop  action  plans  to  resolve  them.  Data  on  runway  incursion  incidents 
can  be  used  to  determine  whether  the  use  of  action  teams  reduces  such 
incidents  at  the  airports  in  question.  Finally,  runway  incursion  incidents 
occur  with  sufficient  frequency  to  make  it  possible  to  measure  progress 
between  evaluations.  Several  hundred  runway  incursion  incidents  have 
been  reported  each  year  this  decade. 

The  Safer  Skies  team  working  on  approach  and  landing  accidents  in 
commercial  aviation  is  considering  using  an  accident  precursor  to  evaluate 
the  effectiveness  of  training  and  other  related  interventions.  The  team 
determined  that  unstable  aircraft  approaches  to  airports  were  clearly 
precursors  to  many  approach  and  landing  accidents.5  Several  problems  can 
contribute  to  unstable  approaches,  including  excess  speed  on  approach, 
aircraft  flaps  not  in  position,  and  an  approach  that  is  too  steep  or  too 
shallow.  Data  on  each  of  these  key  aspects  are  recorded  on  an  aircrafts 
flight  data  recorder.  Thus,  the  team  has  an  opportunity  to  develop  a 
performance  measure  based  on  reducing  the  number  of  unstable 
approaches. 

Unstable  approaches  have  some  features  of  good  performance  measures. 
First,  they  are  measurable.  Second,  data  on  them  can  be  obtained  from 
flight  data  recorders.  However,  there  are  barriers  to  obtaining  these  data 
that  must  be  overcome  before  unstable  approaches  can  be  used  as  a 


^Nonetheless,  runway  incursions  are  now  underreported  because  FAA  does  not  keep 
statistics  on  runway  incursions  at  airports  without  towers  because  there  are  no  air  traffic 
controllers  present  to  report  the  incidents. 

Unstable  approaches  can  lead  to  loss  of  control,  landing  short  of  the  runway,  and 
overrunning  the  runway,  among  other  problems.  An  approach  can  become  unstable  for  any 
of  the  following  reasons:  late  air  traffic  control  clearance  to  descend,  late  notification  of  the 
landing  runway,  late  selection  of  the  landing  configuration,  rapidly  changing  weather,  poor 
prior  planning  by  the  crew,  or  a  pilot’s  misjudgment  of  the  circumstances. 
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performance  measure  for  approach  and  landing  interventions.  Finally, 
unstable  approaches  occur  frequently  enough  to  measure  progress 
resulting  from  interventions. 


Potential  Barriers  Exist  to 
the  Use  of  Some 
Accident  Precursors  as 
Performance  Measures 


Barriers  exist  to  using  some  accident  precursors  as  performance  measures. 
For  example,  the  use  of  unstable  approaches  as  a  performance  measure 
depends  on  access  to  information  from  aircraft  flight  data  recorders.  While 
some  airlines  use  data  from  flight  recorders  to  analyze  the  causes  of  safety 
problems  on  routine  flights,  there  are  barriers  to  sharing  this  information 
with  FAA  or  with  other  airlines.  Logistical  barriers  include  (1)  the  limited 
information  tracked  by  older  flight  data  recorders  still  in  use  and  (2) 
differences  in  the  ways  that  air  carriers  have  programmed  flight  data 
recorders  to  track  key  information.  Because  of  these  differences,  the  kinds 
of  data  items  needed  to  track  unstable  approaches  are  not  being  captured 
with  enough  consistency  for  this  measure  to  be  a  good  indication  of 
performance  throughout  commercial  aviation. 
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Other  potential  barriers  also  prevent  the  use  of  unstable  approaches  as  a 
performance  measure.  Among  these  barriers  are  the  ongoing  debate  about 
how  data  from  flight  recorders  are  to  be  shared,  who  should  have  access  to 
these  data,  and  whether  legal  enforcement  cases  can  be  initiated  on  the 
basis  of  these  data.  Numerous  major  aviation  safety  reports  in  this  decade 
have  advocated  a  program  that  would  gather  and  analyze  information  from 
flight  data  recorders  about  routine  flights.  FAA  has  for  years  promised  to 
establish  such  a  program.6  However,  the  inability  of  FAA,  the  aviation 
industry,  and  other  federal  agencies  to  reach  consensus  on  key  aspects  of 
this  program  has  delayed  its  finalization.  While  shared  data  can  move 
safety  forward,  concerns  about  potential  litigation,  criminal  indictments, 
and  the  violation  of  an  air  carrier  employee’s  privacy  have  served  as 
barriers  to  the  establishment  of  the  program.  Such  concerns  have  also 
delayed  the  finalization  of  other  programs  to  enhance  the  sharing  of 
aviation  safety  data.  For  example,  safety  reports  have  for  years 
recommended  the  establishment  of  Aviation  Safety  Action  Programs  to 
encourage  voluntary  self-reporting  of  safety  violations  by  pilots;  FAA 
issued  an  advisory  circular  providing  guidance  for  these  programs  on 
March  17,  2000.7 


Conclusions  Most  Safer  Skies  teams  have  not  finished  analyzing  the  causes  of  the  safety 

problems  they  are  working  on  and  have  not  yet  selected  interventions  to 
prevent  the  problems.  Thus,  these  teams  have  not  developed  methods  to 
evaluate  the  effectiveness  of  their  interventions.  But  when  interventions 
have  been  selected,  most  have  been  implemented  without  first  determining 
how  to  evaluate  their  effectiveness.  Neither  FAA  nor  the  aviation  industry 
will  have  the  information  that  will  be  critical  in  determining  whether  the 
interventions  have  made  progress  in  resolving  the  safety  problems  until 
appropriate  performance  measures  are  developed.  Evaluating  the  impact 
of  safety  interventions  depends  on  having  good  baseline  data  on  the  extent 
of  the  problem  prior  to  the  implementation  of  the  intervention,  explicit 


6The  program  that  FAA  has  promised  to  implement,  the  Flight  Operational  Quality 
Assurance  program,  uses  flight  data  to  detect  technical  flaws,  unsafe  practices,  or 
conditions  outside  of  desired  operating  procedures  early  enough  to  allow  timely 
intervention  to  avert  accidents  or  incidents. 

7An  Aviation  Safety  Action  Program  is  a  partnership  program  between  FAA  and  the  aviation 
industry  to  encourage  voluntary  self-reporting  by  pilots  of  safety  violations.  The  program 
provides  some  protection  from  enforcement  actions  for  inadvertent  violations  in  return  for 
valuable  data  that  can  be  used  to  analyze  safety  problems. 
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short-  and  long-term  goals  against  which  to  measure  progress,  and 
performance  measures  that  are  clearly  linked  to  the  safety  problem  being 
addressed.  In  addition,  as  Safer  Skies  teams  select  interventions  to  address 
the  safety  problems  that  caused  fatal  aviation  accidents,  it  would  be  useful 
to  identify  clearly  any  existing  barriers  to  the  development  of  performance 
measures.  These  barriers  include  differences  in  aircraft  equipment  and  the 
absence  of  needed  data.  Once  such  problems  are  clearly  identified,  FAA 
and  the  aviation  industry  can  work  jointly  to  resolve  them. 


Recommendations  To  improve  the  ability  to  determine  the  effectiveness  of  Safer  Skies 

interventions,  we  recommend  that  the  Secretary  of  Transportation  direct 
the  FAA  Administrator  to  work  with  the  Safer  Skies  steering  committees  to 
direct  the  teams  to  identify  the  extent  of  fatal  accidents  resulting  from  the 
safety  problems  they  are  working  on.  If  possible,  data  should  be  developed 
to  establish  a  consistent  baseline  against  which  to  measure  the  progress 
that  results  from  the  Safer  Skies  initiative.  If  an  analysis  team  has  already 
completed  its  report,  the  implementation  team  working  on  the  same  safety 
problem  should  develop  these  baseline  data.  More  specifically,  to  better 
measure  progress  toward  overall  safety  goals,  we  recommend  that  the  FAA 
Administrator  work  with  the  Safer  Skies  steering  committees  to  revise  the 
implementation  guidance  to  (1)  develop  an  overall  performance  measure 
or  measures  to  determine  progress  toward  eliminating  the  safety  problem 
the  team  is  addressing;  (2)  consider  using  accident  precursors  as 
performance  measures  for  the  safety  problem  in  question;  and  (3)  identify 
any  barriers  that  may  impede  the  implementation  of  performance 
measures. 


Agency  Comments 


DOT  and  FAA  officials  agree  in  principle  with  the  need  for  baseline  data  on 
the  extent  of  each  safety  problem  and  performance  measures  to  determine 
progress  toward  overall  safety  goals.  They  concur  with  the  potential  of 
accident  precursors  as  possible  performance  measures  and  with  the 
importance  of  identifying  any  barriers  that  may  impede  the  implementation 
of  performance  measures. 
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FAA  coordinated  extensively  with  numerous  representatives  from  the 
aviation  industry,  other  federal  agencies  involved  in  aviation  safety,  and  its 
own  staff  on  the  identification  of  safety  problems  and  the  selection  of 
interventions.  However,  efforts  to  prioritize,  fund,  and  evaluate  Safer  Skies 
initiatives  could  be  better  coordinated  with  industry  and  within  FAA  and 
the  Department  of  Transportation  (DOT).  Joint  government-industry 
efforts  to  improve  safety  are  not  new,  but  participants  noted  that  the 
initiative  was  more  inclusive  than  prior  joint  efforts.  This  inclusive 
approach  should  help  FAA  gain  consensus  on  which  interventions  will  best 
address  aviation  safety  problems.  However,  our  review  identified  three 
coordination  problems  that  could  undermine  the  implementation  and 
evaluation  of  Safer  Skies  interventions.  First,  although  FAA  officials  have 
repeatedly  committed  to  funding  interventions  agreed  upon  by  all  parties 
working  on  the  initiative,  skepticism  still  exists  among  some  participants 
as  to  whether  this  commitment  can  or  will  be  honored.  This  is  particularly 
true  in  general  aviation.  It  also  remains  unclear  what  process  will  be  used, 
if  funding  is  limited,  to  reprioritize  available  resources  to  ensure  funding 
for  interventions  that  emerge  later  but  have  greater  potential  for  reducing 
the  fatal  accident  rate.  Finally,  Safer  Skies  steering  committees,  FAA,  and 
DOT  have  not  coordinated  how  they  will  measure  progress  in  achieving  the 
accident  reduction  goal  for  commercial  aviation. 


The  Safer  Skies 
Initiative  Involves  an 
Unprecedented  Level 
of  Coordination 
Between  Industry  and 
Government 


FAA  included  aviation  experts  from  a  wide  range  of  government  and 
industry  organizations  on  the  Safer  Skies  steering  committees  and  the 
teams  working  on  the  16  safety  problems.  Many  participants  represent 
groups  that  are  directly  responsible  for  the  nation’s  aviation  safety,  such  as 
the  air  carriers  and  the  manufacturers  of  aircraft  and  engines.  Other 
participants  come  from  trade  associations  that  represent  various  aviation 
groups  or  from  federal  agencies  that  share  responsibility  for  aviation  safety, 
including  the  Department  of  Defense  and  the  National  Aeronautics  and 
Space  Administration.  In  addition,  while  giving  priority  initially  to  reducing 
the  U.S.  accident  rate,  the  initiative  recognized  the  increasingly  global 
nature  of  aviation.  In  an  effort  to  address  both  domestic  and  worldwide 
aviation  safety  problems,  the  commercial  aviation  steering  committee 
included  representatives  from  two  international  aviation  authorities,  the 
Joint  Aviation  Authorities  and  the  International  Civil  Aviation  Organization. 


Joint  efforts  between  industry  and  government  officials  to  study  aviation 
safety  problems  are  not  new.  In  prior  years,  government  and  industry 
convened  various  joint  teams  to  review  aviation  safety  issues  and  make 
recommendations;  however,  according  to  Safer  Skies  participants,  those 
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earlier  teams  did  not  always  include  representatives  from  major 
organizations  who  were  responsible  for  aviation  safety.  As  a  result,  FAA 
was  not  always  successful  in  obtaining  consensus  on  the  safety 
interventions  that  those  teams  recommended.  Safer  Skies  participants 
noted  that  the  level  of  participation  and  cooperation  for  this  initiative  is 
unprecedented  among  the  major  groups  responsible  for  aviation  safety  and 
should  enhance  FAA’s  chances  of  implementing  the  safety  interventions 
made  by  the  various  teams. 

Moreover,  the  initiative  coordinated  ongoing  aviation  safety  work  that  was 
being  conducted  independently  by  FAA,  industry,  and  other  federal 
agencies.  For  example,  aircraft  manufacturers  had  initiated  an  exhaustive 
study  on  ways  to  prevent  uncontained  engine  failure.  FAA  eventually  joined 
the  aircraft  manufacturers  in  this  study,  and  it  subsequently  became  part  of 
the  Safer  Skies  agenda.  In  addition,  the  industry  and  FAA  had  been 
conducting  independent  studies  on  runway  incursions  and  CFIT.  Under  the 
initiative,  representatives  from  the  aircraft  manufacturers,  airline  industry, 
and  government  are  members  of  the  teams  studying  16  safety  problems, 
and  together  they  will  decide  on  the  strategies  to  address  them. 


The  Funding, 
Prioritization,  and 
Evaluation  of  Safer 
Skies  Interventions 
Could  Be  Better 
Coordinated 


While  coordination  between  government  and  industry  organizations 
participating  in  the  initiative  has  been  extensive,  we  identified  three  areas 
in  which  coordination  could  be  improved.  First,  although  FAA  has 
committed  to  funding  interventions  approved  by  the  Safer  Skies  steering 
committees,  uncertainty  remains  about  the  agency’s  ability  to  fund  these 
safety  interventions.  The  steering  committees  for  commercial  aviation  and 
general  aviation  have  both  sought  commitment  to  the  implementation  and 
funding  of  interventions  before  giving  final  approval  to  move  forward. 
However,  FAA’s  commitment  has  come  at  different  points  in  the  approval 
process  for  interventions  recommended  by  these  steering  committees,  and 
FAA’s  commitment  to  the  general  aviation  interventions  was  still  uncertain 
even  after  some  industry  and  FAA  officials  believed  the  steering  committee 
had  given  its  final  approval.  As  a  consequence,  general  aviation  participants 
were  more  skeptical  about  whether  FAA  would  implement  or  fund  their 
safety  interventions.  Second,  it  remains  unclear  what  process  will  be  used 
to  reprioritize  available  resources  if  funding  is  limited.  Finally,  Safer  Skies 
steering  committees,  FAA,  and  DOT  have  not  coordinated  how  they  will 
measure  Safer  Skies’  progress  in  achieving  the  goal  of  reducing  the  fatal 
accident  rate  in  commercial  aviation  by  80  percent  by  2007. 
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earlier  teams  did  not  always  include  representatives  from  major 
organizations  who  were  responsible  for  aviation  safety.  As  a  result,  FAA 
was  not  always  successful  in  obtaining  consensus  on  the  safety 
interventions  that  those  teams  recommended.  Safer  Skies  participants 
noted  that  the  level  of  participation  and  cooperation  for  this  initiative  is 
unprecedented  among  the  major  groups  responsible  for  aviation  safety  and 
should  enhance  FAA’s  chances  of  implementing  the  safety  interventions 
made  by  the  various  teams. 

Moreover,  the  initiative  coordinated  ongoing  aviation  safety  work  that  was 
being  conducted  independently  by  FAA,  industry,  and  other  federal 
agencies.  For  example,  aircraft  manufacturers  had  initiated  an  exhaustive 
study  on  ways  to  prevent  uncontained  engine  failure.  FAA  eventuallyjoined 
the  aircraft  manufacturers  in  this  study,  and  it  subsequently  became  part  of 
the  Safer  Skies  agenda.  In  addition,  the  industry  and  FAA  had  been 
conducting  independent  studies  on  runway  incursions  and  CFIT.  Under  the 
initiative,  representatives  from  the  aircraft  manufacturers,  airline  industry, 
and  government  are  members  of  the  teams  studying  16  safety  problems, 
and  together  they  will  decide  on  the  strategies  to  address  them. 


The  Funding, 
Prioritization,  and 
Evaluation  of  Safer 
Skies  Interventions 
Could  Be  Better 
Coordinated 


While  coordination  between  government  and  industry  organizations 
participating  in  the  initiative  has  been  extensive,  we  identified  three  areas 
in  which  coordination  could  be  improved.  First,  although  FAA  has 
committed  to  funding  interventions  approved  by  the  Safer  Skies  steering 
committees,  uncertainty  remains  about  the  agency’s  ability  to  fund  these 
safety  interventions.  The  steering  committees  for  commercial  aviation  and 
general  aviation  have  both  sought  commitment  to  the  implementation  and 
funding  of  interventions  before  giving  final  approval  to  move  forward. 
However,  FAA’s  commitment  has  come  at  different  points  in  the  approval 
process  for  interventions  recommended  by  these  steering  committees,  and 
FAA’s  commitment  to  the  general  aviation  interventions  was  still  uncertain 
even  after  some  industry  and  FAA  officials  believed  the  steering  committee 
had  given  its  final  approval.  As  a  consequence,  general  aviation  participants 
were  more  skeptical  about  whether  FAA  would  implement  or  fund  their 
safety  interventions.  Second,  it  remains  unclear  what  process  will  be  used 
to  reprioritize  available  resources  if  funding  is  limited.  Finally,  Safer  Skies 
steering  committees,  FAA,  and  DOT  have  not  coordinated  how  they  will 
measure  Safer  Skies’  progress  in  achieving  the  goal  of  reducing  the  fatal 
accident  rate  in  commercial  aviation  by  80  percent  by  2007. 
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Skepticism  Persists  About 
FAA’s  Ability  to  Fund  Safety 
Interventions 


Skepticism  persists  about  whether  FAA  can  or  will  be  able  to  honor  its 
commitments  to  fund  the  interventions  approved  by  the  Safer  Skies 
steering  committees  to  reduce  the  fatal  accident  rate.  This  is  especially  true 
in  the  general  aviation  community.  This  skepticism  results  partly  because 
the  process  for  approving  and  funding  Safer  Skies  interventions  has 
worked  differently  for  general  aviation  than  it  has  for  commercial  aviation 
thus  far.  This  has  contributed  to  differing  perceptions  about  the  likelihood 
of  the  funding  and  implementation  of  interventions.  These  perceptions 
have  resulted  in  part  from  the  different  processes  used  by  the  two  steering 
committees  to  seek  approval  and  funding  from  participating  organizations, 
from  the  way  interventions  have  moved  forward  within  these  two  Safer 
Skies  committees,  and  from  FAA’s  handling  of  the  interventions 
recommended  by  them. 


The  Process  for  Final  Approval 
of  Interventions  Has  Worked 
Differently  in  the  Two  Steering 
Committees 


The  final  approval  of  recommended  safety  interventions  has  worked 
differently  in  the  commercial  aviation  and  general  aviation  steering 
committees.  The  commercial  aviation  steering  committee  has  documented 
its  process  for  approving  interventions,  which  involves  members’  gaining 
the  approval  of  their  respective  organizations  for  both  implementation  and 
funding.  This  approval  comes  in  two  stages.  First,  steering  committee 
members  brief  their  respective  organizations  on  the  general  concept  of 
each  intervention  under  consideration  and  seek  preliminary  approval  of 
each  intervention.  Changes  and  modifications  may  be  suggested  by  the 
organizations.  For  organizations  that  will  be  involved  in  the 
implementation  of  an  intervention,  the  preliminary  approval  also  involves  a 
tentative  commitment  to  fund  the  cost  of  implementing  any  interventions 
for  which  they  are  responsible.  Once  members  grant  preliminary  approval, 
the  steering  committee  asks  the  team  to  draw  up  detailed  implementation 
plans  for  each  intervention.  These  implementation  plans  are  then 
submitted  to  the  steering  committee  for  the  next  level  of  approval. 
Members  subsequently  seek  final  approval  of  these  plans  from  the 
organizations  they  represent,  including  firm  resource  and  funding 
commitments  if  appropriate.  When  participating  organizations  concur  with 
the  detailed  implementation  plans,  the  steering  committee  grants  final 
approval.  To  date,  most  of  the  commercial  aviation  teams  have  forwarded  a 
few  interventions  at  a  time  for  final  approval  by  the  steering  committee, 
rather  than  complete  lists  of  interventions  to  address  multiple  aspects  of 
complex  safety  problems,  such  as  CFIT.  Thus,  when  the  commercial 
aviation  steering  committee  has  given  its  final  approval  for  an  intervention, 
members  interviewed  told  us  they  assumed  that  the  intervention  had  a  high 
priority  and  that  implementation  would  take  place  because  the 


Page  88 


GAO/RCED-OO-1 1 1  Safer  Skies  Initiative 


Chapter  5 

Coordination  Has  Been  Extensive  but  Needs 
Improvement  for  the  Safer  Skies  Initiative  to 
Succeed 


organizations  responsible  for  implementation  had  already  committed  both 
the  staff  and  funding  needed. 

In  contrast,  the  general  aviation  steering  committee  had  not  documented 
its  process  for  approving  interventions  at  the  time  of  our  review,  although  it 
recently  developed  draft  procedures,  according  to  FAA’s  response  to  our 
draft  report.1  Furthermore,  commitment  to  provide  resources  for  them  is 
still  pending,  although  some  members  of  both  FAA  and  industry  who  serve 
on  the  steering  committee  understood  that  final  approval  had  been  given  to 
the  interventions  chosen  to  address  CFIT  and  weather.  Once  these  two 
implementation  teams  submitted  their  draft  reports  to  the  general  aviation 
steering  committee,  the  steering  committee  asked  members  to  have  their 
organizations  review  and  comment  on  each  intervention.  This  process 
resulted  in  preliminary  approval  or  disapproval  of  the  concept  of  each 
intervention,  in  some  cases  after  the  intervention  was  modified. 
Organizations  responsible  for  the  implementation  of  interventions  also 
were  expected  to  give  a  tentative  commitment  to  fund  the  cost  of  their 
implementation.  The  steering  committee  then  asked  the  teams  to  develop 
detailed  implementation  plans  for  each  intervention  and  to  submit  those 
for  its  final  approval.  These  two  teams  recommended  and  developed  plans 
for  a  total  of  17  interventions,  many  of  which  involve  subinterventions  and 
will  require  substantial  resources  either  in  the  form  of  staff  or  funding  from 
FAA.  Because  of  the  number  and  potential  cost  of  interventions  contained 
in  the  two  general  aviation  reports,  FAA  requested  that  the  general  aviation 
steering  committee  prioritize  the  interventions.  The  general  aviation 
steering  committee  prioritized  the  interventions  in  the  letter  that 
transmitted  the  final  CFIT  and  weather  reports  to  the  FAA  Administrator  in 
March  2000.  Unlike  the  commercial  aviation  teams,  which  have  presented 
one  intervention  at  a  time  to  the  steering  committee,  the  general  aviation 
teams  have  presented  their  complete  series  of  interventions  for  each  safety 
problem. 

As  the  general  aviation  CFIT  and  weather  reports  moved  toward  final 
approval,  however,  confusion  arose.  Some  industry  and  FAA  participants 
believed  that  these  reports  had  received  final  approval.  This  perception  is 
supported  by  a  March  22,  2000,  letter  from  the  industry  and  FAA  co-chairs 
of  the  general  aviation  steering  committee  transmitting  to  the  FAA 
Administrator  the  final  CFIT  and  weather  implementation  reports  with 


'We  did  not  have  an  opportunity  to  review  the  general  aviation  steering  committee’s  draft 
procedures. 
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their  detailed  implementation  plans.  The  letter  and  accompanying  reports 
identified  high-priority  interventions  for  immediate  implementation.  These 
participants  were  concerned  because  FAA  was  still  undecided  which 
interventions  would  actually  be  implemented  and  funded.  In  contrast, 
FAA’s  informal  written  comments  in  response  to  our  draft  report  state  that 
final  approval  has  not  been  given  to  either  implementation  report  and 
depends  on  the  completion  of  detailed  implementation  plans  by  the  FAA 
offices  responsible  for  carrying  out  the  implementation.  According  to  the 
Director  of  Aircraft  Certification,  confusion  arose  because  some  members 
of  the  steering  committee  had  “misperceptions”  about  what  levels  of 
approval  had  been  agreed  to. 

FAAs  internal  review  and  funding  process  for  Safer  Skies  interventions  has 
led  to  uncertainty  about  whether  some  interventions  will  be  funded,  in  part 
because  interventions  forwarded  by  the  commercial  aviation  and  general 
aviation  steering  committees  have  been  handled  somewhat  differently  thus 
far.  Like  the  other  organizations  participating  in  the  initiative,  FAA  must 
commit  its  own  resources  to  the  interventions  that  it  is  responsible  for 
implementing.  In  October  1999,  FAA  formed  an  executive  council  to  help 
coordinate  the  implementation  of  the  agency’s  safety  agenda,  including 
how  to  provide  funding  and  staff  resources  for  Safer  Skies  interventions. 
The  executive  council  includes  the  heads  of  each  of  FAAs  major  program 
offices,  its  general  counsel,  and  a  regional  administrator.  The  executive 
council  has  not  yet  documented  its  process  for  approving  and  funding 
interventions,  however,  and  it  remains  unclear  at  what  point  FAA  is 
committing  resources  to  implement  Safer  Skies  interventions.  This 
uncertainty  has  led  to  different  perceptions  on  the  part  of  some  FAA  and 
industry  participants  about  the  likelihood  that  interventions  will  be 
implemented  and  funded. 

FAA  staff  working  on  the  initiative  described  differences  in  the  way  the 
executive  council  has  handled  interventions  proposed  by  the  two  steering 
committees.  These  differences  have  resulted  in  a  clear  indication  of 
funding  for  commercial  aviation  interventions  before  that  steering 
committee’s  final  approval  is  given,  while  the  general  aviation  steering 
committee’s  final  approval  was  given  on  a  series  of  weather  and  CFIT 
interventions  that  have  yet  to  be  approved  and  funded  by  FAA.  When 
proposed  Safer  Skies  interventions  are  under  serious  consideration  by  the 
steering  committees,  they  are  also  presented  to  the  executive  council  for 
discussion  of  their  possible  impact  on  workload  and  budget,  according  to 
FAA  staff  who  serve  as  co-chairs  of  the  two  steering  committees.  The 
executive  council  provides  feedback  to  the  steering  committees  before 
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interventions  are  approved.  FAA  staff  serving  on  Safer  Skies  committees 
presented  conflicting  views,  however,  of  when  FAA  commits  to  funding 
interventions.  Several  of  the  FAA  staff  interviewed  said  that  FAA’s 
commitment  of  staffing  and  funding  to  commercial  aviation  interventions 
occurs  before  that  steering  committee  gives  its  final  approval  to 
interventions. 

However,  the  Director  of  FAA’s  Aircraft  Certification  Service,  who  serves  as 
co-chair  of  the  commercial  aviation  steering  committee,  described  the 
executive  council’s  role  as  having  more  room  for  interpretation  of  the 
intervention  and  a  subsequent  determination  of  whether  funding  is 
available.  She  said  that,  once  the  intervention  is  approved,  the  executive 
council  again  discusses  it,  determines  whether  to  accept  it  as  stated  or  to 
modify  it,  assigns  it  to  an  FAA  office  for  implementation,  and  determines 
how  it  fits  in  with  the  office’s  existing  priorities.  The  program  office  then 
reviews  the  intervention,  can  suggest  modifications  that  will  achieve  the 
same  goal,  and  determines  whether  the  intervention  can  be  accomplished 
with  existing  resources  or  requires  a  request  for  additional  funding.  She 
said  that  the  executive  council  could  also  request  that  the  steering 
committee  modify  or  prioritize  interventions.  For  example,  she  said  that 
FAA  agreed  to  implement  the  commercial  aviation  CFIT  team’s 
recommendation  to  develop  precisionlike  airport  approaches,2  concluded 
that  the  agency’s  resources  would  not  permit  the  completion  of  approaches 
for  all  airports  in  the  time  frame  envisioned  by  the  intervention,  and  is  now 
working  with  the  steering  committee  to  identify  which  airports  present  the 
greatest  risks  and  should  be  completed  first.  Similarly,  she  said  that  the 
council  asked  that  the  general  aviation  steering  committee  approve  a 
different  way  to  accomplish  one  intervention  without  hiring  additional  staff 
and  prioritize  its  list  of  CFIT  and  weather  interventions  according  to  which 
ones  will  have  the  most  impact  on  improving  safety  and  reducing  fatalities. 
Because  the  executive  council’s  role  is  new  and  its  procedures  remain 
undocumented,  confusion  persists  about  when  FAA  commits  its  resources 
to  implementing  the  safety  interventions  approved  by  the  steering 
committees.  For  example,  although  FAA’s  executive  safety  council  had 
agreed  in  principle  to  the  highest  priority  interventions  to  address  general 
aviation  safety  problems  caused  by  weather  and  CFIT,  FAA’s  response  to 
our  draft  report  indicated  that  final  approval  and  funding  depend  on  the 
completion  of  detailed  implementation  plans.  As  a  consequence,  several 


2Precisionlike  approaches  improve  aviation  safety  by  enabling  all  flight  crews  and  aircraft  to 
fly  a  stabilized  vertical  path  to  the  runway  end  for  all  instrument  approach  procedures. 


Page  91 


GAO/RCED-OO-111  Safer  Skies  Initiative 


Chapter  5 

Coordination  Has  Been  Extensive  but  Needs 
Improvement  for  the  Safer  Skies  Initiative  to 
Succeed 


Safer  Skies  participants  from  FAA  and  industry,  especially  those  working 
on  general  aviation  issues,  expressed  some  concern  about  whether  the 
recommended  interventions  would  be  funded  or  implemented. 

These  concerns  stem  partly  from  FAA’s  past  record  for  implementing  safety 
recommendations.  FAA’s  budget  does  not  specifically  identify  and  commit 
resources  to  implementing  Safer  Skies  interventions.  For  example,  FAA  has 
no  funds  set  aside  in  its  budgets  for  fiscal  years  2000  or  2001  for  general 
aviation  interventions.  However,  FAA’s  Deputy  Associate  Administrator  for 
Regulation  and  Certification  said  that  the  agency’s  approach  to  budgeting  is 
to  retain  flexibility  by  not  identifying  specific  budget  amounts  for  such 
efforts  as  the  Safer  Skies  initiative.  While  we  do  not  advocate  including 
specific  Safer  Skies  line  items  in  FAA’s  budget,  the  uncertainty  about 
funding  and  implementation  also  exists  because  FAA  has  either  not  fully 
funded  or  not  implemented  some  safety  recommendations  in  the  past. 
Several  industry  participants  in  the  initiative  specifically  mentioned 
concerns  about  FAA’s  lack  of  follow  through  on  safety  recommendations  to 
decrease  the  number  of  runway  incursions.  Although  FAA  has  received 
many  recommendations  for  reducing  runway  incursions,  continuing 
problems  in  this  area  have  been  partially  attributable  to  insufficient  funding 
of  the  safety  plans  FAA  developed,  according  to  DOT’s  Inspector  General.3 
Additionally,  after  initially  planning  to  fund  the  agency’s  new  inspection 
system,4  FAA  has  still  not  provided  funding  to  hire  analysts  to  review 
inspection  data  on  the  nation’s  10  major  airlines  for  possible  safety 
concerns.  While  FAA  has  implemented  many  safely  recommendations  over 
the  years,  concerns  still  persist  about  the  agency’s  ability  to  fund  new 
safety  initiatives.  Greater  assurance  about  the  implementation  of  Safer 
Skies  interventions  could  be  provided  in  two  ways.  First,  as  mentioned  in 
chapter  3,  stronger  mechanisms  for  tracking  the  implementation  of 
interventions  from  all  three  steering  committees  need  to  be  established. 
Second,  clarifying  FAA’s  process  for  committing  resources  for 
implementing  interventions  would  provide  greater  assurance  of  their 


3Federal  Aviation  Administration:  Aviation  Safety,  DOT  Inspector  General  Report  No.  AV- 
1999-069  (Statement  of  Alexis  M.  Stefani,  Deputy  Assistant  Inspector  General  for  Aviation, 
U.S.  Department  of  Transportation  before  the  Subcommittee  on  Transportation  and  Related 
Agencies,  Committee  on  Appropriations,  U.S.  House  of  Representatives,  Mar.  10, 1999). 

’We  presented  our  findings  and  recommendations  about  the  new  inspection  system,  the  Air 
Transportation  Oversight  System,  in  Aviation  Safety:  FAA's  New  Inspection  System  Offers 
Promise,  but  Problems  Need  to  Be  Addressed  (GAO/RCED-99-183,  June  28,  1999). 
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implementation.  Both  of  these  steps  would  improve  coordination  between 
FAA  and  other  Safer  Skies  participants. 

Thus  far,  the  interventions  approved  by  steering  committees  have  not 
required  a  major  commitment  of  time  and  resources  by  either  FAA  or 
industry  groups.  But  future  interventions  may  require  substantial  resources 
not  included  in  FAA's  current  budget,  and  choices  may  have  to  be  made 
about  which  interventions  to  fund.  Furthermore,  FAA  addresses  and  funds 
many  issues  beyond  those  on  the  Safer  Skies  agenda,  including  security 
issues  and  improvements  to  the  air  traffic  control  and  airport 
infrastructure.  FAA’s  executive  council  provides  a  forum  for  agency 
managers  to  discuss  and  prioritize  program  and  resource  needs.  However, 
without  clear  priorities  and  a  unified  aviation  safety  agenda  that  also  takes 
such  issues  into  account,  FAA  will  continue  to  address  aviation  piecemeal, 
rather  than  as  an  integrated  system.  While  the  Safer  Skies  initiative 
represents  a  major  step  in  the  direction  of  coordinating  the  nation’s 
aviation  safety  agenda,  a  more  far-reaching  effort  has  not  yet  been 
undertaken  to  coordinate  the  nation’s  complete  aviation  agenda. 


The  Initiative  Does  Not 
Have  A  Process  for 
Prioritizing  Interventions  to 
Ensure  the  Implementation 
of  Those  With  the  Greatest 
Potential  to  Reduce  the 
Fatal  Accident  Rate 


The  initiative  has  not  developed  a  process  for  prioritizing  interventions  to 
ensure  the  implementation  of  those  with  the  greatest  potential  to  reduce 
the  fatal  accident  rate  if  funding  is  limited.  The  initiative  has  involved 
prioritization  at  several  points  thus  far.  First,  the  teams  addressing  safety 
problems  in  commercial  aviation  and  general  aviation  have  prioritized  the 
interventions  they  considered.  For  example,  the  general  aviation  weather 
team  considered  numerous  possible  safety  interventions  and  eventually 
developed  a  list  of  17  that  it  presented  in  order  of  priority.  The  steering 
committees  have  also  prioritized  interventions.  For  example,  the 
commercial  aviation  steering  committee  has  moved  quickly  on  several 
interventions  that  the  CFIT  implementation  team  considered  as  having  a 
high  priority  and  potential  for  effectiveness.  At  the  request  of  the  executive 
council,  the  general  aviation  team  created  a  unified  list  to  prioritize  its 
CFIT  and  weather  interventions.  Given  the  constraints  of  FAA’s  budget, 
such  prioritization  is  critical  to  ensuring  that  funds  are  expended  on  the 
interventions  that  will  be  most  effective  in  reducing  the  fatal  accident  rate. 


The  ability  to  reprioritize  resources  for  Safer  Skies  interventions  and  other 
aviation  work  may  also  become  critical.  The  Safer  Skies  team  has  just 
begun  work  on  loss  of  control — the  safety  problem  that  caused  the  greatest 
number  of  fatal  accidents  in  commercial  aviation  in  1988-97.  Interventions 
to  address  loss  of  control  are  thus  likely  to  be  critical  for  reducing  the  fatal 
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accident  rate.  If  funding  is  limited,  this  may  mean  reprioritizing  funding 
from  existing  programs  and  Safer  Skies  interventions  that  have  already 
been  approved  to  those  with  more  potential  to  reduce  the  fatal  accident 
rate  and  save  lives.  The  initiative’s  success  will  depend  in  part  on  its  ability 
to  identify  those  interventions  with  the  most  potential  impact  and  to 
prioritize  their  implementation  and  funding.  Safer  Skies  steering 
committees  and  FAA’s  executive  council  have  not  yet  established  any 
process  for  reprioritizing  interventions  if  funding  is  limited. 


Safer  Skies  Steering 
Committees,  FAA,  and  DOT 
Have  Different  Ways  of 
Measuring  Progress  in 
Reducing  Commercial 
Aviation’s  Fatal  Accident 
Rate 


A  lack  of  coordination  among  Safer  Skies  steering  committees,  FAA,  and 
DOT  has  resulted  in  their  having  different  ways  of  measuring  whether  the 
goal  of  reducing  the  fatal  accident  rate  for  commercial  aviation  by  80 
percent  is  achievable  by  2007.  DOT  is  responsible  for  setting  safety  goals 
for  all  modes  of  transportation  under  its  authority,  including  aviation. 
Generally,  FAA  and  other  agencies  under  DOT  have  established  specific 
goals  and  use  measurements  that  evaluate  their  progress  in  meeting  those 
goals  that  are  in  line  with  those  set  by  DOT.  But  currently,  DOT  and  FAA 
measure  progress  toward  the  goal  of  an  80-percent  reduction  in  the  fatal 
accident  rate  for  commercial  aviation  in  different  ways.  DOT’S 
Performance  Plan  for  fiscal  year  2001  establishes  goals  for  reducing  the 
fatal  accident  rate  in  commercial  aviation  that  rely  on  the  Safer  Skies 
initiatives.  To  determine  the  progress  made  in  reducing  the  rate,  DOT’S  plan 
uses  aircraft  flight  hours  as  the  activity  measure.  In  contrast,  the 
commercial  aviation  steering  committee  and  FAA  use  aircraft  departures  as 
the  measure  of  aviation  activity.  Because  DOT,  FAA,  and  Safer  Skies  all 
share  a  common  goal  of  reducing  the  fatal  accident  rate,  consistency  would 
be  desirable  in  the  aviation  activity  measure  they  use  to  calculate  the 
progress  being  made  toward  that  goal.  Since  most  commercial  aviation 
accidents  occur  during  takeoff  and  landing,  we  believe  that  using 
departures  would  better  measure  the  effectiveness  of  the  Safer  Skies 
interventions  for  commercial  aviation. 


Conclusions  Additional  steps  need  to  be  taken  to  ensure  that  those  safety  interventions 

most  critical  to  reducing  the  nation’s  fatal  accident  rate  are  given  top 
priority  and  funding.  If  FAA’s  process  for  prioritizing  and  funding  Safer 
Skies  interventions  is  not  clarified,  there  is  no  assurance  that  the  agency 
will  be  able  to  implement  these  interventions.  If  funding  is  limited,  a 
process  may  well  be  needed  for  reprioritizing  available  staffing  and  funding 
to  ensure  that  the  interventions  with  the  greatest  potential  for  reducing  the 
nation’s  fatal  accident  rate  and  saving  lives  are  implemented  first.  Even  if 
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Safer  Skies  steering  committees  and  FAA  agree  on  the  priorities  for  the 
nation’s  safety  agenda,  these  priorities  will  continue  to  compete  for 
resources  with  other  aviation  needs  until  FAA  develops  a  unified  aviation 
agenda.  Finally,  FAA,  the  Safer  Skies  commercial  aviation  steering 
committee,  and  DOT  are  not  using  the  same  aviation  activity  measure  to 
calculate  the  progress  of  Safer  Skies  interventions  in  reducing  the  fatal 
accident  rate  for  commercial  aviation.  Consequently,  they  may  reach 
different  conclusions  about  the  effectiveness  of  the  Safer  Skies 
interventions  in  achieving  the  goal  of  reducing  the  fatal  commercial 
aviation  accident  rate  by  80  percent  by  2007. 


Recommendations  To  ensure  the  implementation  of  the  Safer  Skies  safety  interventions,  we 

recommend  that  the  Secretary  of  Transportation  direct  the  FAA 
Administrator  to  clarify  the  executive  council's  process  for  committing  to 
the  funding  and  implementation  of  interventions  and  coordinate  with  the 
Safer  Skies  steering  committees  about  the  meaning  and  timing  of  this 
commitment.  To  ensure  that  the  interventions  with  the  greatest  potential 
for  reducing  the  fatal  accident  rate  and  improving  aviation  safety  receive 
needed  resources,  we  recommend  that  the  Secretary  of  Transportation 
direct  the  FAA  Administrator  to  ensure  that  the  executive  council  has  a 
process  in  place  for  reprioritizing  interventions  if  funding  is  limited. 

To  ensure  that  the  extent  of  progress  toward  reducing  the  fatal  accident 
rate  for  commercial  aviation  is  measured  consistently,  we  recommend  that 
the  Secretary  of  Transportation  ensure  that  DOT,  FAA,  and  the  Safer  Skies 
commercial  aviation  steering  committee  all  use  departures  as  the  activity 
measure  for  calculating  the  rate. 


AgenCV  Comments  DOT  and  FAA  officials  concurred  with  our  recommendations  to  clarify  the 

°  J  executive  council’s  process  for  committing  to  the  funding  and 

implementation  of  interventions  and  to  use  departures  as  the  activity 
measure  for  calculating  the  fatal  accident  rate  in  commercial  aviation.  They 
disagreed  with  our  recommendation  that  FAA’s  executive  council  should 
develop  a  process  for  reprioritizing  interventions  if  funding  is  limited.  The 
officials  said  that  such  reprioritization  falls  under  the  agency’s  normal 
processes  for  reprogramming  funding.  However,  the  role  of  the  executive 
council  is  to  help  coordinate  the  implementation  of  the  agency’s  safety 
agenda — including  how  to  provide  funding  and  staff  resources  for  Safer 
Skies  interventions.  We  believe  that  it  would  be  useful  for  the  executive 
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council  to  establish  some  basic  criteria  and  processes  for  evaluating  and 
comparing  the  potential  impact  of  existing  and  emerging  safety 
interventions.  For  this  reason,  we  did  not  modify  or  withdraw  our 
recommendation. 
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